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I   INTRODUCTION 

A.  PREFACE. 

A  primary  goal  of  this  thesis  is  to  analyze  the  Longbow 
HELLFIRE  system,  the  Hardware  in  the  Loop  simulation 
methodology  and  the  statistical  properties  of  lot  acceptance 
testing  for  the  Army  Material  Command  (AMC)  .  This  methodology 
will  be  compared  to  the  existing  HELLFIRE  II  "Fly  to  Buy" 
methodology  that  has  been  in  effect  for  the  last  several 
years . 

B.  SYSTEM  DESCRIPTION. 

1.   Missile  System  Description. 

The  Longbow  HELLFIRE  Modular  Missile  System  (LBHMMS) 
is  an  integral  part  of  the  AH-64D  helicopter  weapon  system, 
which  is  designed  to  defeat  multiple  armored  ground  targets 
and  several  selected  air  targets,  through  the  process  of  radar 
acquisition  targeting  and  ground  engagement  (see  Figure  #1) . 
The  LBHMMS  consists  of: 

•  Longbow  HELLFIRE  Modular  Missile. 

•  Longbow  HELLFIRE  Launcher. 

•  Longbow  Missile  Container. 

•  Longbow  Training  Missile. 

•  Radome  Environmental  Cover. 


FIRE  AND  FORGET  HELLRRE 
MMW  FCR  AND  SEEKER  (LOBL/LOAL) 
ALL  WEATHER  CAPABILITY 
GREATER  RANGE  THAN  HELLFIRE 
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Figure  1   Longbow  HELLFIRE  System  Description 


In  addition,  a  missile  telemetry  sub-system  will  be 
designed  and  used  during  flight  testing  of  several  of  the 
Engineering  and  Manufacturing  Development  (EMD)  Longbow 
missiles1 . 

Tactical  Missile  Description.  The  LBHMMS  consists  of  an  active 
Millimeter  Wave  (MMW)  radar  guidance  section  mated  to  a 
HELLFIRE  II  missile  bus.  The  Longbow  missile  warhead  is  a 
High  Explosive  Anti-Tank  (HEAT)  weapon  that  employs  a  radar 
aided  inertial  guidance  system.  The  bus  consists  of  a  warhead 
section,  a  propulsion  section,  and  a  control  section  (see 
Figure  #2) .  The  LBHMMS  is  capable  of  either  a  Lock-On-Bef ore- 
Launch  (LOBL)  Mode  or  a  Lock-On-After-Launch  (LOAL)  Mode  which 
describes  the  two  methods  in  which  a  target  can  be  engaged  by 
the  Longbow  Missile.  These  are  the  major  components  and 
functions  that  will  be  referred  to  through  out  this  thesis. 
2.   Launch  Platforms. 

The  AH-64D  Helicopter  is  the  primary  launch  platform 
envisioned  for  the  LBHMMS.  The  LBHMMS  will  provide  the  AH-64D 
and  other  airborne  launch  platforms  with  a  fire-and- forget 
capability  for  engaging  targets  handed  over  from  the  Longbow 
Fire  Control  Radar  (FCR)  or  its  functional  equivalent  systems, 
such  as  the  Tactical  Air  Designation  System  (TADS) ,  Airborne 
Target  Handoff  System  (ATHS)  and  the  Integrated  Helmet  and 


1   For  additional  information  concerning  the  Longbow 
HELLFIRE  program  and  points  of  contact  see  Appendix  A. 
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Pigure  2   Longbow  HELLFIRE  Tactical  Missile 


Display  Site  System  (IHADSS) .  Future  variants  may  also 
include  a  ground  launch  platform  in  addition  to  the  airborne 
platforms  described  above. 

C .   BACKGROUND . 

For  the  past  several  decades,  the  U.S.  Army  has  required 
virtually  all  existing  tactical  missile  systems,  be  purchased 
only  after  a  representative  sample  of  missiles  from  each  lot, 
the  production  lot,  has  been  successfully  tested  to  confirm 
specified  performance  requirements.  The  purpose  of  such  lot 
acceptance  testing  plans  is  to  provide  a  qualitative 
assessment  of  missile  system  reliability  for  the  entire  lot 
and  to  provide  an  accept/reject  decision  rule  for  the  entire 
lot  of  missiles.  Under  this  procedure,  the  contractor  only 
delivers  and  gets  paid  for,  those  lots  that  pass  this 
acceptance  test. 

The  "Fly  to  Buy"  acceptance  methodology,  by  its  very 
nature,  requires  that  these  sample  missile  systems  be  fired  in 
a  scenario  that  represents  the  actual  or  anticipated 
requirement  that  the  missile  system  would  be  likely  to 
encounter  under  actual  battlefield  conditions.  This 
inevitably  results  in  the  missile  being  destroyed  while 
verifying  that  it  is  functioning  properly. 

Until  recently,  the  technical  complexity  of  the  missile 
system  and  the  cost  associated  with  performing  "Fly  to  Buy" 
testing  was  not  large  enough  to  cause  rethinking  of  the 


process.  However,  increasing  costs  of  missile  systems  and 
declining  defense  resources  are  forcing  a  re-evaluation  of 
this  type  of  destructive  lot  acceptance  testing.  Other  more 
cost-effective  testing  methods  must  be  found  to  determine  the 
quality  of  missile  lots. 

One  possible  alternative  to  this  problem  is  to  assess  the 
missile's  quality  through  tests  involving  computer 
simulations.  This  process  known  as  Hardware -in -the -Loop 
(HWIL)  testing,  attempts  to  simulate  the  flight  of  the  missile 
without  actually  requiring  it  to  be  fired  or  destroyed.  In 
this  case  the  actual  sample  missile  would  be  attached  to  a 
computer  via  a  cable  tied  to  as  many  of  the  key  components  of 
the  system  as  safely  practical.  A  computer  simulation  of  an 
actual  acceptance  test  scenario  could  then  be  run  that 
exercises  the  missile  in  a  manner  similar  to  an  actual  live 
firing  against  a  specified  target.  The  computer  collects  the 
data,  analyzes  them  and  assesses  the  results  in  terms  of  the 
lot  acceptance  criteria. 

No  specific  Government  methodology  exists  that  addresses 
the  prospect  of  simulating  missile  fights  via  this  HWIL 
process  for  lot  acceptance.  Currently  there  is  no  other  Army 
millimeter  wave  HWIL  facility  which  can  accommodate  "live" 
missile  rounds.  This  would  require  the  construction  of  a 
separate  facility  that  could  perform  this  type  of  testing. 


D.  PURPOSE. 

The  purpose  of  this  thesis  is  to  explore  and  evaluate  the 
current  Longbow  HELLFIRE  Lot  Acceptance  Plan  against  the 
conventional  "Fly  to  Buy"  methodologies  to  determine  what 
advantages  have  been  gained  using  a  HWIL  simulation  for  this 
process  and  what  still  remains  to  be  accomplished  in  order  to 
use  this  new  process  effectively. 

E.  APPROACH. 

Because  the  Longbow  HELLFIRE  Quality  Assurance  Lot 
Verification  Test  (QALVT)  plan  is  a  one  of  a  kind  test  plan 
that  still  is  in  its  draft  form,  it  will  be  necessary  to 
consider  past  plans  that  preceded  it  to  fully  understand  and 
appreciate  the  advantages  it  presents. 

The  approach  used  in  conducting  this  analysis  involves 
first  providing  the  background  information  leading  up  to  the 
establishment  of  an  HWIL  simulation  method.  This  will  be 
discussed  and  provided  in  Chapters  I  &  II.  Next  the 
traditional  "Fly  to  Buy"  methodology  will  be  discussed  in 
Chapter  III,  to  provide  the  reader  with  a  basis  of 
understanding  of  the  key  requirements  involved  in  conducting 
prior  lot  acceptance  testing  and  the  whole  lot  acceptance 
procedure  in  general.  This  will  also  serve  to  outline  the 
development  of  the  HWIL  simulation  method. 

The  current  draft  HWIL  Simulation  Plan  will  be  introduced 
in  Chapter  IV,  and  its  differences  highlighted  from  its 


predecessor  the  "Fly  to  Buy"  methodology.  In  Chapter  V  the 
HWIL  simulation  method  is  analyzed  to  quantify  its  benefits 
and  deficiencies.  A  conclusion  to  the  analysis  is  provided  in 
Chapter  VI.  Finally,  in  Chapter  VII,  suggested  modifications 
and  alternatives  (including  possible  contractual  language)  of 
the  HWIL  methodology  are  discussed. 


II   THE  NEED  FOR  HARDWARE  IN  THE  LOOP  (HWIL)  ACCEPTANCE 
TESTING 


A.   BACKGROUND. 

In  FY  1992,  the  Air- to-Ground  Missile  System  (AGMS) 
Project  Office,  which  includes  HELLFIRE,  HELLFIRE  II,  and 
Longbow  HELLFIRE,  solicited  a  proposal  from  the  United  States 
Army  Test  and  Experimentation  Command,  Redstone  Technical  Test 
Center  (USATECOM  RTTC)  to  develop  an  alternative  method  for 
performing  lot  acceptance  testing  on  Longbow  HELLFIRE  Missiles 
during  Low- Rate  Production  and  subsequent  Full -Rate 
Production.  Prior  lot  acceptance  testing  had  been  performed 
exclusively  through  the  use  of  "Fly  to  Buy"  (FTB)  methodology, 
where  a  lot  sample  of  4  to  10  missiles  per  month  were  flight 
tested  at  Eglin  AFB  to  determine  acceptance  or  rejection  for 
the  remainder  of  the  missiles  in  each  lot.  In  the  intervening 
years  between  the  production  of  HELLFIRE  II  and  the 
development  of  Longbow  HELLFIRE,  there  has  been  a  significant 
rise  in  the  unit  cost  of  each  missile  round.  The  conventional 
"Fly  to  Buy"  lot  acceptance  program  would  require  considerable 
additional  resources.  An  HWIL  simulation  based  testing 
program  will  reduce  the  lot  acceptance  testing  costs. 
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B.  UNITED  STATES  ARMY  MISSILE  COMMAND   (USAMICOM)   POLICY  # 
702-5. 

The  Commanding  General  of  USAMICOM  has  directed  that 
weapon  system  acquisition  planning  will  consider  the  principal 
items  expressed  in  Quality  Assurance  Lot  Verification  Testing 
(QALVT)  before  making  a  final  determination  as  to  what  type  of 
requirements  will  be  included  in  the  final  missile  lot 
acceptance  plan.  The  contents  of  this  directive  was  set  forth 
in  MICOM  POLICY  No.  702-5,  dated  16  May  1990. 

The  application  of  this  policy  is  based  on  three  principal 
decision  factors:  1)  unit  cost  of  missile  less  than  or  equal 
to  $  200k,  2)  production  rate  of  at  least  50  units  per  month, 
3)  estimated  QALVT  less  than  5%  of  the  procurement  cost  (see 
Figures  #  3,  4,  &  5,  U.S.  Army  QALVT  decision  Criteria)2. 

C.  ASSUMPTIONS  AND  TRADE-OFFS. 

The  Air  to  Ground  Missile  System  (AGMS)  Project  Office  and 
USATECOM  RTTC  have  performed  an  analysis  of  the  perspective 
factors  and  trade-offs  involved  in  implementing  a  HWIL  lot 
acceptance  program  versus  a  traditional  "Fly  to  Buy"  (FTB) 
program. 

This  analysis  considered  the  two  alternatives  available  to 
Longbow  HELLFIRE  (LBHF)  as  well  as  the  historical  data 
provided  by  the  earlier  HELLFIRE  I.   In  all  cases,  it  was 
assumed  that  all  FTB  flight  testing  would  be  conducted  at 


US  Army  Missile  Command  Policy  #702-5,  16  May  1990 
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Figure  3    MICOM  Policy  #702-5,  Decision  Criteria 
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QUALITY  ASSURANCE  LOT  VERIFICATION  TESTING  (QALVT) 

CRITERIA  FOR  MEETING     "DIAMOND  A"     (WHICH  ALLOWS  PROGRESSION 

TO  THE  NEXT  STEP  IN  THE  QALVT  LOGIC  DIAGRAM) 

1 .  QUALITY  PROGRAM  IN  ACCORDANCE  WITH  MIL-Q-9858. 

2.  GOVERNMENT  APPROVED  STATISTICAL  PROCESS  CONTROL  (SPC). 

3.  SUPPLIER  RATING  PROGRAM  IN  ACCORDANCE  WITH  MIL-STD-1535. 

4.  LAST  PRODUCT  ORIENTED  SURVEY  (POS)  SCORED  GREATER  THAN  8.0. 

5.  NO  CATEGORY  1  QUALITY  DEFICIENCY  REPORT  (QDR)  RECHVED  IN 
THE  LAST  12  MONTHS. 

6.  SIX  OR  LESS  CATEGORY  2  QDRs  RECEIVED  IN  THE  LAST  12  MONTHS. 

7.  NO  MAJOR  HARDWARE  DESIGN  CHANGES  (SUCH  AS  PRODUCT 
IMPROVEMENT  PROGRAMS)  SINCE  THE  START  OF  QALVT  STEP  1 . 

8.  TEN  OR  FEWER  MAJOR  OR  CRITICAL  RFDs/RFWs  DURING  LAST  12 
MONTHS. 

9.  NO  MAJOR  CHANGES  IN  FACILITIES,  TOOLING,  TEST  EQUIPMENT  OR 
PRODUCTION  PROCESSES  SINCE  THE  START  OF  QALVT  STEP  1 . 

10.  RECEIVING  INSPECTION  YIELDS  GREATER  THAN  93  PERCENT  DURING 
THE  LAST  SIX  MONTHS. 

1 1 .  IN-PROCESS  INSPECTION  YIELDS  GREATER  THAN  93  PERCENT  DURING 
THE  LAST  SIX  MONTHS. 

12.  INTERNAL  CONTRACTOR  QUALITY  AUDIT  DATA  ARE  AVAILABLE  TO 
THE  GOVERNMENT  AND  CONTRACTOR  AGREES  TO  UNANNOUNCED 

GOVERNMENT  AUDITS. 

13.  MISSILES  ARE  WARRANTED. 

14.  NO  POST- ACCEPTANCE  FLIGHT  FAILURES  OF  WARRANTED  MISSILES. 

NOTE:    ITEMS  1  THROUGH  14  ABOVE  ARE  PROVIDED  FOR  GUIDANCE  AND 
MAY  BE  MODIFIED  OR  CHANGED  THROUGH  CONTRACT  NEGOTIATION. 
Figure  5  MICOM  Policy  702-5  Decision  Criteria  (cont.) 
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Eglin  Air  Force  Base  (EAFB) ;  Alabama.   The  primary  factors 
inthe  development  of  the  assumptions  involved  in  this  analysis 
process  -were  as  follows: 
Unit  Costs: 

•  HELLFIRE  cost  per  missile $   20k 

•  Longbow  HELLFIRE  (LBHF)  cost  per  missile $  300k 

@  Low  Rate  Initial  Production  (LRIP) . 

Test  Programs : 

•  HELLFIRE  Lot  Acceptance  Testing:  (FTB)  @  4-10 
missiles  per  month. 

•  Longbow  HELLFIRE  (LBHF)  Lot  Acceptance  Testing: 

(FTB)  @  4-10  missiles  per  month. 

•  Longbow  HELLFIRE  HWIL  Simulation:  4  live  fires  per  year 
and  up  to  20  missiles  simulated  flight  testing  per  month. 

Facilities : 

•  Facility  for  the  HWIL  Simulation:  $  5.8  million  (a  one 
time  cost) .   The  actual  building  and 
instrumentation. 

Labor: 

•  Personnel  needed  to  run  this  facility  for  one  year: 
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estimated  at  approximately   $0.8  million. 

With  these  factors  in  mind,  a  trade-off  analysis  was 
performed  using  costs  for  four  missiles  with  the  results 
portrayed  in  Table  2-1  Production/Acceptance  Analysis3. 

The  total  savings  projected  under  a  HWIL  simulation  lot 
acceptance  test  program  would  represent  cost  savings  of  $13.55 
million  per  year  over  a  conventional  "Fly  to  Buy"  program. 
Even  including  the  $5.8  million  necessary  for  providing  for 
the  HWIL  simulation  facility,  the  payback  period  under  this 
assumption  would  be  less  than  one  year. 


Table  2-1, 
( 

PRODUCTION/ACCEPTANCE  ANALYSIS, 
in  $  millions  per  year) 

HELLFIRE  FTB 

LBHF  FTB 

HWIL  SIM. 

MISSILE  COST1 

$1.15 

$14.40 

$1.20 

EAFB  SUPPORT 

$0.72 

$0.72 

$0.06 

RTTC  SUPPORT 

$0.53 

$0.53 

$0.44 

SYS.  SIM  SUPPORT 

$0.00 

$0.00 

$0.40 

TOTAL 

$2.40 

$15.65 

$2.10 

Notes:  1.  HWIL  SIM. 

missile  costs 

represent  the 

;  cost  of  hook- 

up  and  processing  of  sample  missiles  through  the 
computer  test  facility  and  returning  them  to  their 
original  configuration. 

The  HELLFIRE  FTB  column  indicates  approximately  what  AGMS 

pays  per  year  for  the  HELLFIRE  I  "Fly  to  Buy"  program,  and 

also  indicates  where  the  true  cost  growth  in  FTB  actually 

lies,  namely,  missile  cost.    This  total  amount  includes 


3   Johnson,  J.,  Memorandum  for  AGMS,  "All  Up  Round  Test 
Facility",  18  April  1994. 
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missile  cost,  range  support,  and  pre- flight  support.  The 
Longbow  HELLFIRE  "Fly  to  Buy"  column  indicates  what  the  cost 
would  be  if  the  AGMS  Office  implemented  a  similar  "Fly  to  Buy" 
program  with  just  four  missiles  per  month,  even  though  this 
method  might  require  up  to  ten  missiles  to  complete. 

D.   HWIL  SIMULATION  REQUIREMENTS. 

The  requirements  imposed  on  the  HWIL  facility  by  AGMS  are 
as  follows: 

•  Functional  checkouts  of  hardware  and  software  inter- 
operability of  the  missile. 

•  Test  must  provide  a  confidence  equal  or  better  than  FTB, 
that  missile  lots  are  good  or  bad. 

•  Must  be  non-destructive. 

•  Must  perform  testing  at  the  All -Up-Round  (AUR)  level. 

•  Must  consider  environmental  conditioning. 

•  Number  of  required  missiles  must  be  significantly  lower. 

Testing  at  the  All -Up-Round  level,  refers  to  conducting  a 
test  at  the  complete,  assembled  system  level.  AUR  would  not 
allow  components  of  the  system  to  be  disassembled  to 
accommodate  testing. 

The  facility  required  for  HWIL  testing  would  also  have  to 
have  specific  requirements.  These  would  include  the  following 
items : 

•  The  HWIL  facility  be  located  at  Redstone  Arsenal 


16 


(RSA) . 

•  Capable  of  Vibration  &  Environmental  testing. 

•  Capable  of  handling  a  Longbow  HELLFIRE  All  Up  Round  (AUR) . 

•  Meet  safety  requirements. 
Provide  cost  effective  operation. 

E.   THE  SIMULATION/TEST  ACCEPTANCE  FACILITY. 

The  Simulation/Test  Acceptance  Facility  (STAF)  is  the 
Hardware- in- the  Loop  facility  being  developed  at  RSA  by  the 
U.S.  Army  Test  and  Evaluation  Command  (USATECOM)  Redstone 
Technical  Test  Center  (RTTC)  and  the  U.S.  Army  Missile  Command 
(USAMICOM)  and  System  Simulation  and  Development  Directorate 
(SSDD)  .  This  facility  is  the  only  facility  in  the  U.S. 
capable  of  performing  nondestructive  HWIL  tests  on  "live" 
production  and  developmental  missiles  at  the  millimeter  wave 
frequencies.  A  diagram  of  the  facility  is  shown  at  Figure  6. 
This  facility  is  currently  under  construction  at  RSA  on  the 
north-eastern  side  of  Test  Area  #1.  This  facility  includes 
the  following  equipment,  buildings  and  hardware: 

1.   Bunker. 

This  structure  is  approximately  2,000  square  feet  that 
features  a  test  item  room,  a  test  chamber  and  a  computer  room. 
The  test  item  room  contains  the  compact  missile  test  set  and 
the  environmental  chamber  for  conditioning  missile  rounds 
prior  to  testing.  The  test  chamber  contains  the  missile  under 
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SIMULATION/TEST  ACCEPTANCE  FACILITY  (STAF) 
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Figure  6   Simulation/Test  Acceptance  Facility  Layout 
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test,  the  Three  Axis  Rotational  Flight  Simulator  (TARFS) , 
anechoic  chamber,  and  the  target  generator  horns  of  the 
millimeter  wave  target  simulator.  The  computer  room  contains 
the  target  generation  equipment  and  instrumentation,  telemetry 
instrumentation,  simulation  computer,  and  the  data  analysis 
terminals  (for  inert  rounds) .  There  will  be  a  roll -up  door  in 
this  structure  facing  down  range  that  will  allow  actual  target 
acquisition  and  tracking,  if  desired,  down  the  long  axis  of 
the  TA-l  range.   This  will  allow  the  actual  tracking  of  a 
variety  of  real  world  target  vehicles  with  actual  production 
missiles,  at  ranges  up  to  approximately  6km. 

2.  Target  Generator. 

The  target  generator  system  will  intercept  the 
millimeter  wave  signal  transmitted  by  the  missile,  delay  the 
signal  in  time  to  simulate  the  range  to  the  target,  tap  delay 
the  signal  to  simulate  target  range  extent,  place  proper 
doppler  shift  on  the  signal  to  compensate  for  relative 
movement  and  return  the  signal  to  the  missile  in  real-time  on 
a  pulse  for  pulse  basis. 

3.  Three  Axis  Rotational  Flight  Simulator  (TARFS). 

The  TARFS  will  provide  a  mounting  structure  for  the 
missile  in  the  test  chamber.  It  will  additionally  provide 
real-time  missile  flight  motion  in  pitch,  yaw,  and  roll  to 
simulate  the  missile's  actual  fly  out  to  the  target  along  its 
own  trajectory.   The  missile  fin  position  will  be  monitored 
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and  fed  back  into  the  simulation  in  order  to  control  the  TARFS 
position. 

4.  Telemetry. 
All  missile  and  STAF  data  will  be  collected  and  time 

tagged  to  allow  for  a  real  time  "quick  look"  and  post  mission 
data  analysis. 

5.  The  Simulation  Computer  System. 

The  simulation  computer  system  will  consist  of  two 
computer  systems;  a  control  computer  and  a  modeling  computer. 
The  control  computer  will  run  the  6  Degrees  Of  Freedom  (6-DOF) 
program,  facility  control  software,  perform  inputs  and  outputs 
to  the  modeling  computer  and  interacts  with  the  user.  The 
modeling  computer  simulates  the  complex  signature  of  the 
target  subject  using  parallel  processing  techniques. 

6.  Compact  Missile  Test  Set. 

This  test  set  will  perform  the  majority  of  the  open 
loop  testing  and  characterization  of  each  missile  round  prior 
to  entering  into  simulation  testing.  This  test  set  will 
expose  obvious  flaws  or  defects  in  each  missile  prior  to  this 
testing  and  prevent  wasting  valuable  simulation  time  on 
defective  rounds. 
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Ill   THE  EXISTING  "PLY  TO  BUY"  METHODOLOGY 

A.   INTRODUCTION. 

The  purpose  of  the  "Fly  to  Buy"  Methodology  is  to 
determine  within  an  acceptable  level  of  statistical 
confidence,  that  the  Government  is  purchasing  (in  lot  size 
quantities)  a  product  that  is  reasonably  reliable  and  defect 
free.  In  the  case  of  missile  procurement,  this  is  usually 
accomplished  after  destructively  testing  the  lot  sample  and 
comparing  the  results  against  pre-established  acceptance 
criteria.  The  sample  size  required  to  accomplish  this  task  is 
related  to  the  lot  size  and  the  confidence  interval  required. 
Since  most  missile  systems  tend  to  be  quite  expensive,  this 
can  be  very  costly  in  the  long  run.  Consequentially,  a 
sequential  sampling  plan  is  usually  developed  that  reduces  the 
average  number  of  missiles  tested  yet  satisfies  the 
requirements.  These  plans  are  divided  into  several  stages 
which  specify  a  certain  quantity  and  an  acceptance/rejection 
criteria.  Often  these  plans,  depending  on  the  lot  size  and 
confidence  required,  will  specify  that  testing  be  continued 
and  that  another  stage  of  testing  be  conducted  up  to  a  pre- 
determined quantity.  At  the  end  of  the  final  stage,  a 
decision  will  be  made  to  accept  or  reject  the  lot  and  no  other 
testing  will  be  continued  for  the  lot. 
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B.   THE  HELLFIRE  LOT  ACCEPTANCE  PROGRAM. 

1.  Purpose. 

The  purpose  of  the  HELLFIRE  Lot  Acceptance  Program  is 
to  provide  confidence  that  HELLFIRE  missiles  meet  the 
specified  performance  and  reliability  requirements  of  the 
production  contract4. 

2.  Background. 

a.  Sampling  Plan  Characteristics . 

This  plan  is  intended  to  provide  a  method  for 
performing  lot  acceptance  by  prescribing  an  Acceptable  Quality 
Level  (AQL)  for  the  production  process.  This  AQL  is  used  to 
determine  the  sample  size  for  the  sampling  inspection.  The 
AQL  is  the  largest  allowable  percent  of  defective  items  of  a 
satisfactory  process  average.  This  sampling  plan  and  its  AQL 
are  chosen  in  accordance  with  the  risk  assumed,  as  defined  in 
the  contract.  Thus,  the  AQL  is  the  designated  value  of 
percent  defective  for  which  lots  will  be  accepted  most  of  the 
time  by  the  sampling  procedure  to  be  used.  The  sampling  plan 
is  usually  arranged  so  that  the  probability  of  rejecting  a  lot 
when  the  process  average  is  at  the  designated  AQL  value  is  not 
larger  than  a  preassigned  small  value  (eg.  0.10) .  This  is  the 
producer's  risk.  The  producer's  risk  may  vary  slightly  with 
the  sample  size. 


4  The  HELLFIRE  II  Modular  Missile  System  QALVT  Plan, 
USAMICOM,  July  1989,  is  the  primary  source  of  information  for 
this  chapter. 
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Jb.  Operating  Characteristics   Curves. 

In  military  standard  lot  acceptance  testing 
documents,  the  operating  characteristic  curves  (OC  curve)  for 
normal  inspection,  indicate  the  percentage  of  lots  or  batches 
which  may  be  expected  to  be  accepted  under  the  various 
sampling  plans  for  a  given  process  quality.  These  OC  curves 
are  normally  shown  for  single  sampling;  curves  for  double  and 
multiple  sampling  are  matched  as  closely  as  practicable.  The 
OC  curves  shown  for  AQLs  greater  than  10.0  are  based  on  the 
Poisson  distribution  and  are  applicable  for  defects  per 
hundred  units  inspection;  those  for  AQLs  of  10.0  or  less  and 
sample  sizes  of  80  or  less  are  based  on  the  binomial 
distribution  and  are  applicable  for  percent  defective 
inspection;  those  for  AQLs  of  10.0  or  less  and  sample  sizes 
larger  than  80  are  based  on  the  Poisson  distribution  and  are 
applicable  either  for  defects  per  hundred  units  inspection,  or 
for  percent  defective  inspection  (the  Poisson  distribution 
being  an  adequate  approximation  to  the  binomial  distribution 
under  these  conditions) .  Tabulated  values,  corresponding  to 
selected  values  or  probabilities  of  acceptance  (Pa,  in 
percent)  are  given  for  each  of  the  curves  shown.  These  tables 
are  also  given  for  tightened  inspection  processes.  An  example 
of  a  typical  OC  curve  is  shown  at  Figure  7. 
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Figure  7      Typical   OC  Curve   for  Lot  Acceptance   Purposes 
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3.  HELLFIRE  Lot  Acceptance  Definitions  and  Procedures. 

Missile  -  A  missile  shall  consist  of  one  LBHF  High 
Explosive  Anti-Tank  (HEAT)  missile  as  defined  by  Missile 
Specification  (MIS-SPEC)  MIS-44430  or  MIS-42560. 

Test  Article  -  The  test  article  will  be  a  HFLB  HEAT 
production  missile  which  has  been  selected  for  sample  testing. 

Lot  -  A  lot  shall  consist  of  the  production  quantity 
for  a  given  month  which  has  been  produced  under  the  same 
processes.  In  all  cases,  missile  lot  sizes  shall  be  subject 
to  the  constraints  of  homogeneity  with  respect  to  changes  in 
the  processes,  configuration  changes,  deviations  or  waived 
hardware,  etc.  ,  which  may  materially  affect  performance  of  the 
end  item.  The  Government  reserves  the  right  to  combine 
monthly  build  lots  when  necessary/appropriate. 

Lot  Sample  -  A  lot  sample  is  the  quantity  of  ten 
missiles  randomly  selected  from  a  lot  build  of  100  missiles  or 
more  which  will  be  environmentally  conditioned  and  then  tested 
for  the  purpose  of  lot  acceptance  or  rejection.  Each  lot 
sample  of  ten  missile  is  further  divided  into  three  subgroups 
consisting  of  lot  sample  test  article  numbers  1-4,  5-8,  and  9- 
10  respectively. 

4.  Sampling  Procedure. 

As  indicated  above,  in  the  case  of  HELLFIRE,  ten 
missiles  will  be  randomly  selected  from  each  lot  build,  and 
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shall  be  delivered  to  Redstone  Arsenal  (RSA) ,  Alabama,  for 
environmental  conditioning  and  then  shipped  to  Eglin  Air  Force 
Base  (EAFB)  for  the  actual  missile  flight  testing.  The 
Government  then  shall  accept  or  reject  the  entire  lot  of 
missiles  (which  is  anticipated  to  be  roughly  one  month's 
production)  based  on  the  following  sequential  sampling  plan: 

Table  3-1,   SEQUENTIAL  SAMPLING  PLAN. 


No. 

Msls  Tested 

No.  Msl  Failures  (Cumulative) 

Accept    Continue    Reject 
Lot       Testing     Lot 

4 

0         12 

8 

12          3 

10 

2                     3 

In  addition  to  the  ten  missiles  selected  for  testing, 
two  contingency  missiles  shall  be  selected  from  each  lot. 
These  two  contingency  missiles  will  be  used  in  the  event  that 
any  of  the  ten  test  articles  are  damaged  or  otherwise 
unsuitable  for  testing.  This  will  prevent  disrupting  the  test 
process  if  one  or  two  missiles  have  to  be  substituted  for 
these  reasons. 

Smaller  samples  shall  be  selected  from  lots  of  less 
than  100  units  per  month  (a  distinct  possibility  in  the  case 
of  diminishing  procurement  funding  or  program  stretch  outs) . 
The  minimum  number  of  missiles  to  be  selected  under  these 
conditions  is  as  follows  in  Table  3-2  below. 
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Table  3-2,   SMALL  SAMPLE  SIZES 


LOT  SIZE 

LOT  SAMPLE  SIZE 

1-10 

1 

11-25 

2 

26-49 

3 

50-99 

5 

The  Government  shall  identify  to  the  contractor 
missiles  (by  serial  number)  which  have  been  selected  for  QALVT 
testing  at  the  time  the  missiles  have  successfully  completed 
final  inspection  at  the  production  facility. 

After  these  missiles  are  identified  to  the  contractor 
as  test  articles,  the  contractor  will  then  seal  the  missile 
containers  with  contractor  furnished  seals,  and  shipped  to  RSA 
for  testing.  The  contingency  missiles  may  be  held  at  the 
contractor  facility  or  shipped  with  the  test  articles  to  RSA. 

The  missiles  selected  for  lot  sample  testing  shall  be 
counted  as  a  portion  of  the  deliverable  quantity  of  the 
applicable  accepted  lot.  If  any  seals  or  containers  are 
broken,  the  Government  reserves  the  right  to  require 
replacement  of  the  missile  in  the  lot  sample  with  a 
contingency  missile.  The  remainder  of  the  lot  of  missiles 
which  have  not  been  selected  for  testing  at  RSA,  shall  be 
packed  in  missile  containers  and  placed  in  secured  storage 
until  completion  of  the  lot  acceptance  testing,  for  which  they 
are  part,  is  complete.  The  serial  number  of  each  missile  for 
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each  lot  shall  be  provided  to  the  Government  at  the  time  it  is 
placed  into  secured  storage. 

Once  a  selected  test  article  has  been  identified  to 
the  contractor,  no  further  tests,  adjustments,  or  repairs 
shall  be  performed  except  as  authorized  by  the  Government. 
Test  articles  rejected  on  arrival  at  RSA  shall  be 
dispositioned  by  pre- flight  nonconformance  procedures.  The 
contractor  shall  test  missiles  at  RSA  on  the  HELLFIRE  Missile 
Compact  Test  Set  (HMCTS)  in  accordance  with  procedures 
specified  in  the  QAVLT  Plan  and  perform  pre- flight  check  out 
at  EAFB  on  the  HELLFIRE  Compact  Firing  Test  Set .  The 
Government  reserves  the  right  to  omit  any  portion  of  the  lot 
acceptance  test,  including  environmental  tests,  of  any  or  all 
of  the  sample  missiles.  After  Government  inspection  at  RSA, 
test  articles  shall  be  stored  in  a  Government  security  locked 
area. 

5.   Flight  Testing. 

After  Government  inspection  and  acceptance  procedures 
at  RSA  have  been  completed  and  prior  to  shipment  to  EAFB,  the 
lot  sample  missiles  will  be  subjected  to  Captive  Flight 
Vibration  (CFV)  and  one  subgroup  will  be  environmentally 
conditioned  at  RSA,  in  accordance  with  the  lot  acceptance  plan 
and  the  applicable  missile  specifications. 

Upon  completion  of  these  tests  and  environmental 
conditioning  at  RSA,  the  missiles  will  be  transported  to  EAFB 
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for  pre- flight  and  firing  per  Table  3-3. 

TABLE  3-3,   TEST  FIRING  MATRIX 


LOT  # 

TEST  ART 

PROFILE 

MODE 

RANGE 

COND. 

6n+l 

1 

INDIRECT 

LOW 

LOAL 

L 

AMB 

6n+l 

2 

INDIRECT 

HIGH 

LOAL 

L 

AMB 

6n+l 

3 

DIRECT 

LOBL 

M 

AMB 

6n+l 

4 

DIRECT 

LOBL 

M/8° 

AMB 

6n+2 

1 

INDIRECT 

HIGH 

LOAL 

L 

COLD 

6n+2 

2 

INDIRECT 

LOW 

LOAL 

M 

COLD 

6n+2 

3 

DIRECT 

LOAL 

S 

COLD 

6n+2 

4 

DIRECT 

LOBL 

M/8° 

COLD 

6n+3 

1 

INDIRECT 

LOW 

LOAL 

S 

HOT 

6n+3 

2 

DIRECT 

LOBL 

S/8° 

HOT 

6n+3 

3 

DIRECT 

LOAL 

M 

HOT 

6n+3 

4 

INDIRECT 

HIGH 

LOAL 

M 

HOT 

6n+4 

1 

DIRECT 

LOAL 

M 

AMB 

6n+4 

2 

INDIRECT 

HIGH 

LOAL 

M 

AMB 

6n+4 

3 

INDIRECT 

LOW 

LOAL 

S 

AMB 

6n+4 

4 

DIRECT 

LOBL 

S/8° 

AMB 

6n+5 

1 

DIRECT 

LOBL 

M 

COLD 

6n+5 

2 

INDIRECT 

HIGH 

LOAL 

M 

COLD 

6n+5 

3 

INDIRECT 

LOW 

LOAL 

L 

COLD 

6n+5 

4 

DIRECT 

LOBL 

S/8° 

COLD 

6n+6 

1 

DIRECT 

LOBL 

M 

HOT 

6n+6 

2 

DIRECT 

LOBL 

M/8° 

HOT 

6n+6 

3 

INDIRECT 

HIGH 

LOAL 

L 

HOT 

6n+6 

4 

INDIRECT 

LOW 

LOAL 

L 

HOT 

Where  n  =  0,  1,  2,  3 ,  Lock- on  can  be  either 

before  or  after  launch  (LOBL  or  LOAL)  and  Range  which  will 
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be  at  short  (up  to  2km) ,    medium  (2-5)  and  long  (+5)  ,  can 
include  an  8°  offset.   Offset  is  the  deviation  from  the 
straight  line  trajectory  that  the  missile  selects  (either 
right  or  left)  to  help  it  to  distinguish  the  actual  target 
from  possible  background  clutter.   Conditioning  (COND.) 
represents  the  environmental  temperature  conditioning  either 
at  ambient  (AMB) ,  cold  (-45  F)  or  hot  (+145  F) . 

The  procedures  used  to  conduct  the  actual  firing  at 
Eglin  AFB  will  be  controlled  by  that  installation.   Missiles 
will  be  transported,  inspected  and  prepared  for  firing. 
This  will  involve  conducting  a  test  readiness  review  (TRR) 
to  insure  that  all  the  test  requirements,  range  support, 
instrumentation,  and  safety  considerations  have  been 
satisfied  prior  to  actual  missile  firing. 

The  objective  of  this  specified  missile  firings  is  to 
mix  two  ranges,  all  modes,  offsets,  with  varying  degrees  of 
difficulty  on  each  block  of  the  four  firings  so  that  all 
missile  specifications  will  eventually  be  tested  when  this 
table  is  completed.  There  are  a  few  limitations.  They  are: 
1)  Only  two  firing  ranges  (distances)  can  be  instrumented  on 
the  firing  range  at  one  time,  and  2)  All  firing  from  any  one 
lot  will  be  performed  at  the  same  temperature  conditioning  for 
all  test  articles.  The  Government  may  require  the  contractor 
to  fire  at  any  of  the  conditions  in  the  matrix  above  or  any 
combination  of  conditions  that  are  in  accordance  with  the 
appropriate  Military  Specification  (MIL-SPEC) .    Prior  to 
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firing,  the  Government  and  contractor  shall  confirm  that  all 
firing  equipment  and  associated  range  equipment  are  in  proper 
working  condition  and  calibrated. 
6.   Scoring. 

All  test  articles  shall  be  scored  as  either  a:  1) 
Success,  2)  No  Test,  or  3)  Failure,  in  accordance  with  the 
following  criteria. 

a.  Pre- flight  Nonconformance: 

Any  nonconformance  of  a  test  article  or  to  Test 
Design  Package  (TDP)  requirements  discovered  during  inspection 
(including  nonconformances  discovered  prior  to,  during,  or 
after  environmental  testing  of  missiles  at  RSA  or  pre-flight 
checkouts  at  EAFB)  shall  constitute  cause  for  failure  of  the 
test  article  and  may  be  cause  for  lot  rejection.  The 
following  procedure  will  apply  in  determining  the  extent  of 
further  testing  on  the  subject  lot: 

•  Testing  of  a  missile  shall  be  stopped  at  the  point  any 
failure  or  nonconformance  is  discovered  and  a  preliminary 
analysis  shall  be  made  by  the  contractor  and  submitted  to 
the  Government  within  24  hours. 

•  The  Government  will  then  determine  whether  this  missile 
will  continue  through  test  and  be  fired  or  whether  it  will 
be  failed  and  returned  to  the  contractor  for  failure 
analysis  and  future  production. 
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•  If  a  missile  exhibits  a  pre- flight  nonconformance  and  the 
Government  elects  to  continue  testing  that  missile,  that 
particular  pre- flight  nonconformance  shall  be  scored  a  "no 
test". 

•  If  the  Government  determines  from  the  failure  analysis 
that  the  pre- flight  nonconformance  will  or  is  uncertain  it 
will  have  an  impact  on  the  flight  test  parameters,  the 
missile  will  be  scored  a  safety  or  reliability  failure  as 
appropriate. 

Based  on  the  preliminary  analysis  of  the  pre- 
f light  nonconformance,  the  Government  and  contractor  may 
jointly  elect  to  continue  testing  of  the  lot.  A  determination 
to  continue  shall  not  relieve  the  contractor  of  the 
responsibility  to  correct  the  failure  condition  in  all 
missiles,  as  deemed  necessary  by  the  Government. 
Jb.  No  Test 

A  no  test  may  stem  from  a  variety  of  causes  and 
sources  related  to  the  missile  under  test  or  circumstances 
surrounding  these  tests.  They  will  include  areas  resulting 
from: 

•  A  malfunction  of  equipment  other  than  the  test  article  or 
proven  post  launch  environments  which  causes  the  missiles 
to  experience  conditions  outside  of  those  specified  in  the 
Missile  Specification  (MIS-SPEC) . 
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•  Failures  induced  by  Government  personnel  error  or  failures 
induced  by  errors  of  those  authorized  to  act  for  the 
Government,  either  deliberate  or  inadvertent,  or  operation 
beyond  prescribed  limits. 

•  In  the  event  that  the  contractor  aborts  a  missile  launch 
and  the  subsequent  analysis  proves  that  the  missile  did 
not  experience  an  anomaly,  the  missile  will  be  scored  a 
"no  test".  The  contractor  shall  pay  all  retest  costs  and 
the  Government  will  notify  the  contractor  of  the 
acceptance  or  rejection  of  the  lot  within  45  additional 
calendar  days  beyond  the  time  identified  in  the  lot 
acceptance  plan. 

•  Although  a  failure  did  not  occur,  the  missile  displays  a 
property  not  previously  observed  and  is,  therefore,  of 
special  interest.  It  is  anticipated  that,  if  left  in  the 
test  sequence,  this  missile  will  pass  all  tests  and  when 
launched  will  fly  successfully  and  hit  the  target  within 
the  required  accuracy  thus  fulfilling  QALVT  requirements. 
If  the  missile  is  permitted  to  remain  in  the  test  sequence 
all  the  way  through  to  a  successful  launch,  the 
opportunity  to  determine  the  cause  of  this  newly  observed 
property  will  be  conducted  by  the  appropriate  range 
personnel.  If  this  occurs  prior  to  launch,  a 
Government /contract or  decision  shall  be  made  regarding 
further  testing  of  this  missile. 
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c.      Failures. 

Failure:  Any  malfunction  (not  meeting  the  no  test 
criterion)  which  would  preclude  the  missile  from  performing 
within  the  requirements  of  the  MIS-SPEC  will  be  categorized  as 
either  a  safety  or  reliability  failure  as  follows: 

Safety  failures:  A  lot  shall  be  rejected  as  a  result 
of  one  pre- flight  nonconformance  or  missile  firing  that 
results  in  a  condition  that  may  be  hazardous  or  unsafe  as 
defined  in  the  Technical  Design  Package  (TDP)  or  as  defined 
below.   A  safety  failure  will  have  occurred  if: 

•  A  warhead  detonates  prior  to  the  minimum  safe  arm 
distance  from  the  launcher  of  150  meters. 

•  An  inadvertent  launch  occurs  which  is  traced  to  a 
missile  malfunction. 

•  There  is  a  delay  (hang  fire)  of  more  than  five 
seconds  between  the  initiation  of  the  firing 
signal  and  the  ignition  of  the  rocket  motor. 

•  Any  event  that  would  endanger  the  launching 
helicopter  (e.g.  major  debris  thrown  in  the 
area  by  the  missile  upon  launch) . 

Reliability  Failure:  Accept/reject  criteria  for 
reliability  shall  be  attributed  based  on  the  total  number  of 
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failures.  Accept/reject  numbers  shall  be  as  specified  in  the 
sampling  plan  (see  Table  3-1,  Sequential  Sampling  Plan,  pg 
26)  .  A  reliability  failure  shall  have  resulted  when  one  or 
more  of  the  following  occurs: 

•  A  missile  misses  the  target.   A  target  miss  shall 
have  occurred  if  an  intact  missile  does  not  meet 
the  accuracy  requirements  as  stated  in  the 
corresponding  MIS -SPEC. 

•  The  missile  impacts  the  ground  prior  to  impacting 
the  target. 

•  The  warhead  fails  to  detonate  in  accordance  with 
the  timing  requirements  of  MIS-SPEC. 

•  The  warhead  detonates  during  flight  due  to  a 
missile  malfunction  and  is  not  otherwise  scored  a 
safety  failure. 

•  A  no-fire  occurs.   A  no-fire  is  defined  as  failure 
of  the  missile  to  fire  following  application  of 
the  firing  signal. 

•  The  missile  flies  a  profile  other  than  that 
preset . 

•  A  pre- flight  nonconformance  determined  to  be  a 
reliability  failure  in  accordance  with  the  QALVT 
Plan. 
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d.  Success. 

Success;  All  missile  flights  not  scored  as  a  no 
test  or  failure  will  be  scored  as  a  success.  All  available 
test  data  and  flight  hardware  debris  shall  be  made  available 
to  the  contractor  as  soon  as  practical;  visual  examination  of 
impact  debris  and  recovery  of  the  maximum  amount  of  missile 
hardware  debris,  that  can  safely  be  handled,  shall  be  allowed 
prior  to  destruction  of  the  remainder  of  the  hazardous 
components  (if  deemed  feasible  by  the  Explosive  Ordinance 
Disposal  unit  (EOD)  ) ,  contingent  on  approval  by  the  on-site 
EOD  representatives.  Scoring  of  missile  launches  as  success, 
failure,  or  no- test  will  be  accomplished  by  the  Government. 
7.   Acceptance/Rejection  Criteria. 

Acceptance  or  rejection  of  a  lot  is  based  on  pre- 
flight  inspection  and  tests  at  RSA  and  EAFB  and  flight 
acceptance  tests  on  a  sample  of  missiles  randomly  selected  by 
the  Government  from  each  lot  and  scored  in  accordance  with  the 
lot  acceptance  plan.  In  addition  missiles  will  not  be 
accepted  until  all  waivers,  deviations,  Engineering  Change 
Proposals  (ECPs) ,  or  other  issues  affecting  a  lot  are  resolved 
to  the  Government's  satisfaction. 

Upon  accepting  a  lot  of  missiles,  the  contractor  shall 

deliver  to  the  Government  all  remaining  lot  sample  missiles, 

(including  contingency  missiles)  not  flight  tested.  Missiles 

which  have  been  disassembled/repaired  in  failure  analysis  and 
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have  been  subjected  to  lot  acceptance  captive  flight  and 
vibration  testing  will  be  made  functional,  by  completing  a 
functional  baseline  test  at  the  lot  temperature  designated  at 
RSA  and  accepted  for  retirement  at  RSA. 

8.  Resubmittal  Criteria. 

Lots  which  fail  to  pass  the  acceptance  criteria  may  be 
resubmitted  by  the  contractor  unless  an  alternative  plan  of 
action  for  the  lot  acceptance  of  missiles  is  agreed  upon 
between  the  Government  and  contractor.  Failure  analysis  will 
be  conducted  and  deficiencies  corrected  from  the  initial 
submittal  prior  to  the  resubmittal  as  described  above.  A 
failed  lot  may  be  resubmitted  a  maximum  of  two  times.  All 
scoring  and  rejection  criteria  will  remain  the  same  except  for 
adjustments  made  in  the  quantity  of  test  articles  required  for 
decreased  lot  sizes. 

9.  Contractor  Liability. 

When  a  lot  sample  fails  the  acceptance  criteria  and  is 
rejected  by  the  Government  or  when  a  lot  meets  the  acceptance 
criteria  and  one  or  more  failures  occur,  the  contractor  shall 
perform  failure  analysis  and  shall  submit  a  detailed  report 
describing  the  deficiencies  in  performances  which  resulted  in 
rejection  or  failure  and  the  action  necessary  to  correct  and 
prevent  recurrence  of  the  deficiencies.  The  contractor's 
report  shall  be  prepared  in  accordance  with  the  data  item 
reporting  requirements  contained  in  DI-RELI- 80253 . 
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In  the  event  of  a  failed  lot,  additional  lots  shall 
not  be  submitted  for  acceptance  until  corrective  action  has 
been  approved  by  the  Government.  Also,  testing  may  be 
suspended  at  the  Government's  option  on  any  test  articles 
located  at  RSA  or  EAFB. 

10.  Responsibility  for  Retesting  Rejected  Lots. 

In  the  event  lots  are  rejected,  any  and  all  expenses 
associated  with  failure  analysis,  replacement  hardware, 
rework,  reinspection,  packaging,  packing,  handling, 
transportation,  and  retest  shall  be  borne  by  the  contractor. 
The  foregoing  expenses  are  inclusive  of  the  cost  of 
replacement  of  missiles,  Government  Furnished  Equipment  (GFE)  , 
cost  of  range  operations,  environmental  conditioning,  and 
additional  failure  analysis.  With  respect  to  retest  costs 
assuming  a  resubmittal  of  4  missiles,  the  contractor  shall 
reimburse  the  Government  in  the  amount  of  $4,000  for  each 
missile  environmentally  conditioned  at  RSA,  and  $10,000  for 
each  missile  flown  in  each  resubmitted  lot  test  sample.  At 
the  contractor's  option  and  expense,  the  remaining  test 
missiles  from  a  rejected  lot  may  be  returned  to  him. 
Replacement  missiles  will  be  in  the  same  quantity  expended  in 
initial  test,  retest,  and  those  rendered  non-deployable  due  to 
environmental  conditioning  less  the  ten  (10)  missiles 
allocated  for  each  lot  tested  in  the  acceptance  plan  and 
contingency  missiles  that  replace  "no  test"  missiles. 
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11.  Responsibilities. 

a.      Contractor  Representative. 

The  contractor  representative  is  responsible  for 
technical  support  effort  in  observing,  analyzing,  and 
supporting  program  plans  and  procedures  to  meet  program 
milestones  and  objectives.  Additionally,  he  will  coordinate 
with  other  supporting  agencies  to  insure  mission  support 
requirements  are  fulfilled  in  a  timely  manner.  The  contractor 
shall  provide  a  single  individual  at  RSA  and  EAFB  who  shall 
act  as  the  point  of  contact  for  all  decisions  related  to  local 
test  activities.  The  contractor  may  participate  in  all  test 
mission  planning  sessions,  Mission  Readiness  Reviews  (MRR) , 
data  reviews,  and/or  briefings. 

The  contractor  shall  provide  all  missile  round 
information  and  simulation  results  required  to  describe 
expected  flight  performance  of  the  missile.  This  shall 
include  nominal  flight  trajectories  and  expected  three  sigma 
variation  bounds  of  this  trajectory.  The  contractor  may 
monitor  the  launch  and  shall  assist  the  Government  in  solving 
any  problem  areas  which  may  arise  due  to  missile  malfunction. 
In  case  of  a  suspected  missile  "failure"  or  "no 
test"  the  following  activities  shall  be  performed: 

•  The  contractor  shall  perform  a  failure  analysis  on  any 

test  article  which  are  suspected  failures  or  "no  tests". 

Results  of  this  analysis  and  recommendations  for  further 
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activities  shall  be  prepared  in  a  Failure  Analysis  Report 
in  accordance  with  data  item  reporting  requirements 
contained  in  DI-RELI- 80253  . 

•  The  contractor  shall  prepare  a  Corrective  Action  Plan  in 
accordance  with  DI-RELI- 80254  for  all  failures  which 
result  in  the  rejection  of  a  missile  lot.  This  plan  shall 
describe  all  activities  and  actions  planned  by  the 
contractor  to  solve  the  missile  failure  problem. 
Government  approval  of  the  Corrective  Action  Plan  is 
required  prior  to  implementation. 

In  the  case  of  a  rejected  lot,  the  contractor 
shall  develop  and  prepare  a  plan  for  lot  resubmission  in 
accordance  with  DI-RELI- 80254  indicating  the  scheduling  and 
activities  to  be  performed  to  resubmit  the  lot  to  the 
Government  for  acceptance.  Government  approval  of  this  plan 
is  required  prior  to  implementation. 
b.      U.S.   Air  Force. 

U.S.  Air  Force  -  Range  Support  Contractor  &  Range 
Support  Representative: 

The  U.S.  Air  Force  Support  contractor  and 
representative  is  responsible  for  overall  flight  range 
operations  to  include  Data  Acquisition  and  Control  System 
operation,  test  instrumentation,  data  collection  and 
reduction,  missile  ready  storage  and  delivery,  environmental 
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chamber  operation,  range  security,  range  safety,  conduct  of 
the  countdown  from  upload  to  launch,  missile  recovery 
operations,  target  complex  operations,  data  and  documentary 
camera  operation,  laser  spot  data,  and  geodetic  surveys  of 
missile  impact  points. 

U.S.  Air  Force  -  46th  Test  Wing  -  Test 
Engineer /Coordinator: 

The  Air  Force  test  engineer/coordinator  will  be 
responsible  for  overall  Air  Force  coordination,  management, 
and  scheduling  of  resources  to  support  missions  on  Range  C-72, 
the  Longbow  HELLFIRE  flight  test  range. 


U.S.  Air  Force  Range  Systems  Directorate 
Representative : 

The  Range  Systems  Directorate  formally  controls 
all  test  procedures,  instrumentation,  and  software 
configuration.  No  change  or  variation  in  procedures  or 
configuration  will  be  made  without  prior  written  approval  of 
the  Range  System  Directorate  engineering  representative.  All 
questions  concerning  range  instrumentation  or  facility 
performance,  readiness,  and  utilization  will  be  directed  to 
the  Range  Systems  representative. 

No  information  or  data  generated  at  the  facility 
will   be   released   without   the   representative's   Quality 

41 


Assurance  review.  All  requests  for  assistance  or  support  from 
the  Range  Operation  and  Maintenance  (O&M)  contractor  shall  be 
made  through  this  representative.  No  direction  of  the  O&M 
contractor  shall  be  made  by  the  missile  contractor  or  other 
support  personnel . 
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IV   THE  HARDWARE  IN  THE  LOOP  (HWIL)  SIMULATION  METHOD 

A.  INTRODUCTION. 

As  in  the  case  of  many  areas  of  Government  regulation,  lot 
acceptance  has  several  underlying  requirements  that  must  be 
addressed  regardless  of  the  methodology  adopted  to  perform  the 
overall  controlling  action.  This  is  certainly  true  for  the 
Longbow  HELLFIRE  missile.  Since  several  aspects  of  HWIL  will 
not  differ  much  from  those  areas  already  identified  in  the 
"Fly  to  Buy"  methodology,  only  those  areas  that  differ  will  be 
addressed.  Therefore,  the  reader  can  assume  that  all  other 
areas  not  specifically  addressed  will  remain  the  same. 

B.  LONGBOW  HELLFIRE  HWIL  SIMULATION  QALVT. 
1.   Purpose 

The  purpose  of  the  Longbow  HELLFIRE  Quality  Assurance 
Lot  Verification  Test  (QALVT)  program  is  to  provide  a  cost 
effective  means  of  determining  the  acceptability  of  Longbow 
HELLFIRE  production  missiles  while  still  meeting  the  specified 
missile  performance  and  reliability  requirements  stated  in  the 
contract5 . 


5  The  Longbow  HELLFIRE  Modular  Missile  System,  Low  Rate 
Initial  Production  QALVT  Plan,  USAMICOM,  23  May  1994,  is  the 
source  of  information  used  throughout  this  chapter. 
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2.  The  QALVT  Plan 

The  Longbow  HELLFIRE  QALVT  plan  provides  a  method  to 
subject  a  random  sample  from  a  lot  of  Longbow  HELLFIRE 
missiles  to  a  nondestructive,  environmentally  conditioned, 
dynamic,  mechanical,  and  functional  test  through  the  use  of  a 
Hardware  in  the  Loop  Simulation  that  depicts  an  electronic 
target  similar  to  those  that  are  anticipated  to  be  encountered 
under  future  battlefield  conditions.  The  resulting  miss 
distance  would  be  the  primary  performance  measure. 

3.  Definitions. 

Missile  -  A  "missile"  shall  consist  of  one  Longbow 
HELLFIRE  HEAT  missile  as  defined  in  the  MIS. 

Lot  -  A  lot  shall  consist  of  the  production  quantity 
for  a  given  month  which  has  been  produced  under  the  same 
conditions  utilizing  the  same  processes. 

Lot  Sample  -  A  quantity  of  fifteen  missiles  randomly 
selected  from  a  lot  which  will  be  tested  in  accordance  with 
the  QALVT  Plan. 

4.  Lot  Sampling  Procedure. 

Instead  of  the  ten  missiles  required  for  FTB,  fifteen 
missiles  shall  be  randomly  selected  from  each  lot  and 
delivered  to  RSA.  These  fifteen  missiles  shall  have  been 
previously  tested  on  special  inspection  equipment,  to  be 
determined  (TBD) ,  and  shall  have  passed  all  inspections  and 
tests.   The  Government  may  require  the  identification  of  all 
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of  the  contingency  missile (s). 

While  at  RSA,  the  fifteen  missiles  will  be  tested  in 
the  Simulation/Test  Acceptance  Facility  (STAF) .  The 
Government  will  then  accept  or  reject  the  entire  lot  of 
missiles  based  on  the  following  simulation  sampling  plan 
criteria  described  in  Table  4-1  below. 

TABLE  4-1,   SIMULATION  (HWIL)  SAMPLING  PLAN. 


No.  Missiles 
Tested 

Number  of   Missile    Failures 

6 
6 
3 

Accept 

Continue 

Reject 

0 

1 

2 

1 

2 

3 

2 

3 

Procedures  for  selection  and  transportation  of  these  missiles 
to  RSA  is  basically  the  same  as  with  the  "Fly  to  Buy"  method. 
Unless  physically  damaged  in  transportation,  these  missiles 
will  be  determined  to  be  suitable  for  testing  once  they  have 
arrived  and  been  inspected  by  the  Government  representative  at 
the  STAF  facility.  And  as  with  FTB  method,  the  missiles 
selected  for  lot  sample  testing  will  be  counted  as  a  portion 
of  the  deliverable  quantity  of  the  applicable  accepted  lot. 
5.  Test  Sequence  and  Pass  Fail  Criteria. 
a.  Incoming  Inspection. 

There  is  no  pass/fail  requirement  for  this  test 
other  than  the  Government  visually  inspecting  the  missile  for 
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signs  of  damage  due  to  shipment  prior  to  the  actual  testing. 
Jb.  Perform  Portable  Missile  Test  Bench  (PMTB)  Tests. 
This  is  a  system  pre -check  performed  by  the 
contractor  utilizing  portable  test  equipment.  The  actual 
portable  test  equipment  that  will  be  used  to  perform  this  test 
is  anticipated  to  be  fielded  in  conjunction  with  the  missile 
system. 

c.  Perform  Limited  Vibration  Test. 
This  limited  vibration  test  will  be  designed  to 

simulate  some  of  the  vibration  that  the  missile  system  would 
be  exposed  to  if  it  had  been  actually  fired.  The  levels  of 
this  vibration  are  to  be  determined  in  the  future. 

d.  Missile  Preparation. 
Missile  preparation  will  consist  of  removing  the 

precursor  door  and  installing  an  electrical  harness  called  a 
"pigtail",  in  order  to  connect  the  internal  missile  guidance 
electronics  to  the  HWIL  simulator.  Additionally,  it  is 
anticipated  that  an  external  pneumatic  gas  line  will  be 
connected  to  the  fin  mechanism  to  actuate  it  during  the 
simulation  to  demonstrate  that  the  fin  system  is  responding 
properly  to  guidance  commands.  However,  in  order  to 
accurately  assess  this  fin  response,  it  would  be  necessary  to 
replicate  the  aerodynamic  loading  of  the  fins.  This  process 
is  a  very  difficult  task  to  perform  in  a  laboratory 
environment.   Because  of  the  difficulty  involved,  the 
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decision  as  to  whether  or  not  this  will  be  attempted  has  not 
been  made  at  this  time. 

e .   En  vi  ronmen  ta.1    Condi  t±  onlng . 

All  missiles  selected  from  a  lot  shall  be  shipped 
to  Redstone  Arsenal,  but  only  one  subgroup  will  be  initially 
environmentally  conditioned.  The  environment  conditioning 
will  be  for  24  hours.  The  next  subgroup  from  the  lot  will  not 
begin  environmental  conditioning  until  the  previous  subgroup 
has  been  tested  and  the  need  to  test  the  second  subgroup 
established.  The  contractor  may  select  the  temperature  for 
the  first  lot;  thereafter,  the  temperature  conditioning 
sequence  will  typically  continue  as  follows: 

TABLE  4-2,   MISSILE  ENVIRONMENTAL  CONDITIONING. 


LOT  NO. 

SUBGROUP 

SAMPLE  SIZE 

TEMP. 

n  +  1 

1 
2 
3 

6 
6 
3 

AMBIENT 

n  +  2 

1 
2 

3 

6 
6 

3 

COLD 
(-45  F  ) 

n  +  3 

1 
2 

3 

6 
6 

3 

HOT 
(+145  F  ) 

Where  n  =  1, 
2  ,  3  .  .  .  . 

Once   environmental   conditioning   has   been 
completed,  the  missile  will  be  removed  from  the  chamber  and 
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installed  on  a  CARCO6  table  with  shroud.   The  CARCO  table  is 
a  special  test  fixture  that  reduces  the  effect  of  unwanted 
vibration  during  tests  and  allows  the  entire  test  stand  to  be 
moved  within  the  STAF  facility  test  cell. 
f.      Closed  Loop  Electronic  Tests. 

Six  flight  test  scenarios  will  be  simulated  by  the 
STAF7.  Four  scenarios  will  be  against  Stationary  (S)  target 
at  ranges  from  1.5  kilometers  (km)  to  7.5km.  Two  of  the 
stationary  target  scenarios  will  be  conducted  at  ranges  of  1.5 
km  and  2.49  km  in  Terminal  Track  Acquisition  (TTA)  and 
Terminal  Track  (TT)  modes,  both  with  Precision  (P)  Hand  over 
(P  H/0) .  One  will  Lock-on  Before  Launch  (LOBL)  and  the  other 
will  Lock- on  After  Launch  (LOAL) .  The  remaining  two 
stationary  target  scenarios  will  be  conducted  at  ranges  of  6 
km  and  7.5  km  in  modes  Pre-Terminal  Track  Acquisition  (PTA) , 
Pre-Terminal  Track  (PIT) ,  TT  and  Reduced  Radar  Cross  Section 
(RCS) .   Both  will  employ  lock  on  mode  LOAL  and  P  H/0. 

Two  scenarios  will  be  against  moving  (M)  targets 
at  6km  range  and  will  employ  LOBL  lock  on  mode.  One  will 
employ  P  H/0  with  Moving  Target  Acquisition  (MTA)  (spotlight)  , 
PTT  and  TT  modes  while  the  other  employs  Non- Precision  (NP) 
H/0  with  MTA  (scan),  PTT  Ground  Evaluation  (GE),&  TT  modes. 


6  CARCO  is  the  name  of  the  manufacturer  who  produces  this 
test  fixture. 

Specific  tasks  that  will  be  performed  within  the 
closed  loop  tests  are  described  in  Appendix  B. 
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These  scenarios  are  outlined  in  Table  4-3. 

The  requirement  for  success  is  that  each  missile 
shall  be  tested  three  times  (runs)  for  each  scenario.  A 
missile  must  pass  two  runs  out  of  these  three  for  each 
scenario  in  order  to  be  a  success. 

The  miss  distance  requirement  for  each  run  is  2  x 
Circular  Error  Probable  (CEP)  requirements. 

TABLE  4-3,   CLOSED  LOOP  ELECTRONIC  TESTS. 


SCENARIO 

TARGET 

RANGE 

LOCK  ON 

H/0 

MODES  TESTED 

1 

S 

1.5km 

LOBL 

P 

TTA,  TT 

2 

S 

2.49km 

LOAL 

P 

TTA,  TT 

3 

M 

6km 

LOBL 

P 

MTA  spotlight 

PTT,  TT 

4 

M 

6km 

LOBL 

NP 

MTA  scan 

PTT,  GE,  TT 

5 

S 

6km 

LOAL 

P 

PTA,  PTT,  TT 

6 

S 

7.5km 

LOAL 

P 

PTA ,  PTT ,  TT 

The  general  layout  of  this  closed  loop  test  can  be 
seen  at  Figure  8  on  the  following  page. 
g.      Open  Loop  Tests. 

Open  loop  testing  concerns  checking  other  areas  of 
missile  system  performance  that  cannot  be  performed  as  part  of 
the  closed  loop  segment  of  the  HWIL  simulation  itself8. 
These  areas  involve  conducting  the  following  tests  and  tasks: 


8   The  specific  tasks  that  will  be  performed  within  the 
open  loop  tests  are  described  in  Appendix  B. 
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Htrdware-in-the-Loop  Simulated  Flight 
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Figure  8  Closed  Loop  Test  Layout 
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•  RF  Transit  Output  Test  -  in  accordance  with 
Longbow  HELLFIRE  MIS-SPEC-45584 ,  to  within 
minus  4  dB . 

•  Basic  Fin  Response  Test  -  This  is  part  of  the 
PMTB  test  that  will  be  done  prior  to  closed 
loop  tests. 

•  IMS  Performance  -  Move  CARCO  table  and  measure 
gravity- relative  position  +  10%. 

•  Remove  shroud  and  missile  from  CARCO  table. 

•  Remove  pigtail  and  re- install  precursor  door. 

•  Perform  PMTB  test. 

Recertif i cat ion 

Recertif ication  will  be  in  accordance  with 
procedures  that  will  be  determined  at  a  later  date,  but 
involve  insuring  that  the  missile  (after  completing  the  lot 
acceptance  test)  is  placed  back  in  to  the  same  condition  it 
was  in  prior  to  the  start  of  testing. 
h.      Resubmi  tta.1   Criteria. 

Lots  which  fail  to  pass  the  initial  testing  may  be 
resubmitted  by  the  contractor  unless  an  alternative  plan  of 
action  for  the  lot  acceptance  of  missile  is  agreed  upon 
between  the  contractor  and  Government. 

Failure   analysis   shall   be   conducted   and 
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deficiencies  corrected  from  the  initial  submittal  prior  to 
resubmittal  as  outlined  in  the  contractor's  plan  for  lot 
resubmission  which  will  be  prepared  following  the  procedures 
described  in  DI-RELI- 80254 .  A  failed  lot  may  be  resubmitted 
a  maximum  of  two  times. 
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V    ANALYSIS 

A.   INTRODUCTION. 

In  the  process  of  analyzing  the  Longbow  HELLFIRE  HWIL 
Simulation  plan  and  methodology,  several  key  issues  regarding 
the  basic  structure  and  nature  of  the  HWIL  methodology  come  to 
light.  The  first  one  involves  the  very  manner  in  which  the 
test  plan  is  structured.  The  issue  here  being  whether  or  not 
the  concept  of  Hardware  in  the  Loop  testing  needs  to  follow 
the  traditional  structure  typified  by  the  FTB  methodology  or 
even  the  current  HWIL  version.  This  is  because  many  of  the 
reasons  for  the  structure  are  no  longer  important  to  the 
outcome.  Several  factors  need  to  be  discussed  in  view  of  this 
very  pertinent  statement  and  for  several  reasons: 

1)  Since  HWIL  does  not  require  destructive  testing  of  the 
missile,  or  its  components,  the  traditional  cost  saving 
procedures  described  in  earlier  versions  of  the  HELLFIRE 
QALVT  or  similar  sampling  procedures  need  not  necessarily 
apply  to  the  HWIL  simulation. 

2)  The  quantity  of  test  articles  needed  to  perform  HWIL 
testing  need  not  be  limited  to  just  that  of  a  minimum 
quantity  necessary  to  provide  a  desired  statistical 
confidence  that  a  particular  lot  is  good  or  bad. 
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3)  In  order  to  insure  that  this  method  actually  provides  an 
accurate  assessment  of  the  true  nature  of  the  product,  a 
validation  of  the  system/methodology  must  be  conducted  to 
substantiate,  that  the  process  will  provide  the  desired 
information  without  an  increase  in  the  error  probability  or 
that  some  other  undesirable  result  would  occur. 

4)  Actual  missile  flights  will  still  be  conducted  to  gather 
additional  test  data  and  to  provide  a  quality  check  on  the 
entire  system  (to  include  those  areas  that  cannot  be  tested 
using  HWIL,  such  as  the  warhead  detonation  train) .  The  role 
that  these  missile  flight  tests  will  play  in  the  lot 
acceptance  plan  needs  to  be  resolved. 

5)  How  should  the  contract  requirements  for  lot  acceptance 
testing  be  altered  to  take  greater  advantage  of  the  cost 
efficiency  that  HWIL  simulation  could  provide? 

With  these  factors  and  considerations  in  mind,  it  seems 
prudent  that  the  first  step  in  analyzing  the  HWIL  process,  is 
to  quantify/qualify  all  of  the  inputs  and  outputs  of  the  FTB 
and  HWIL  simulation  processes,  and  that  of  the  Longbow 
HELLFIRE  missile  system  that  will  be  tested,  and  then  compare 
these  results  to  the  factors  and  considerations  described 
above.  Finally  these  results  should  be  compared  with  the 
Draft  HWIL  Simulation  QALVT  Plan  to  see  what  modifications 
would  be  warranted.  This  process  should  help  provide  a  basis 
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of  where  or  how  the  current  plan  could  be  modified  to 
accommodate  better  approaches. 

Next  a  method  for  validating  the  HWIL  simulation  should  be 
identified  or  developed. 

B.   THE  TWO  APPROACHES. 

1.   HELLFIRE  (Fly  To  Buy). 

The  original  "Fly  To  Buy"  methodology  was  designed  to 
provide  an  acceptable  level  of  protection  for  the  Government 
and  an  acceptable  level  of  risk  to  the  contractor.  This 
acceptable  level  is  based  on  the  anticipated  reliability  of 
the  system  under  inspection.  In  the  case  of  the  Longbow 
HELLFIRE  system,  this  missile  system  reliability  was 
determined  to  be  0.94,  and  the  acceptable  quality  level  (AQL) 
was  placed  at  0.92.  Considering  the  high  cost  of  the 
HELLFIRE  missile,  the  plan  was  structured  to  achieve  a 
decision  as  to  individual  lot  acceptability  at  the  end  of  the 
least  amount  of  destructive  testing.  In  the  case  of  earlier 
HELLFIRE  missile  systems,  this  led  to  the  development  of  the 
4-4-2  multi-stage  plan  described  in  Chapter  III. 

This  plan  can  be  evaluated  via  an  OC  curve  that 
presents  the  inherent  probability  that  the  lot  of  missiles 
will  be  accepted  based  on  the  test  results  of  the  sample  of 
missiles  tested.  The  OC  curve  for  the  original  FTB  plan  is 
depicted  below  at  Figure  9. 
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4    h    2  SAMPLING  PLAN 
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Figure  9    HELLFIRE  OC  Curve 
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Since  the  prohibitive  cost  involved  in  conducting  this 
type  of  testing  makes  it  unsuitable,  it  is  safe  to  assume  that 
except  for  the  AQL,  no  other  factors  are  pertinent  to  the 
overall  QALVT  issue. 

2.   The  Longbow  HELLFIRE  QALVT  Plan. 

The  Longbow  HELLFIRE  QALVT  Plan,  for  reasons  similar 
to  the  those  for  the  HELLFIRE  FTB  and  the  desire  to  take 
advantage  of  HWIL  ability  to  exercise  the  missile  under  all 
six  of  the  scenario  conditions,  has  a  6-6-3  multi-stage  plan. 
This  plan  has  the  proviso  that  each  missile  will  be  run 
through  each  test  scenario  three  times  and  that  each  missile 
is  required  to  pass  at  least  two  of  the  three  scenario  trials 
in  order  to  pass  any  given  scenario.  Failure  is  still 
determined  by  whether  or  not  each  missile  successfully  passed 
all  six  scenarios  and  whether  or  not  the  overall  sample 
successfully  passed  the  acceptance  test.  This  statement  also 
holds  true  regardless  of  where  the  acceptance  determination  is 
made,  either  during  the  first,  second  or  third  stages  of 
testing.  The  specific  characterisitics  of  the  6-6-3  plan  and 
the  resulting  OC  curve,  by  stage,  can  be  seen  in  Figure  10. 

The  other  major  features  of  the  HWIL  QALVT  Plan,  that 
need  to  be  discussed  are  the  scenarios  themselves.  Because 
all  of  the  Longbow  HELLFIRE  missiles  must  be  run  through  each 
of  the  six  scenarios,  one  might  be  concerned  that  one 
particular  scenario  might  be  more  difficult  than  the  others  to 
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Figure  10    Longbow  HELLFIRE  HWIL  OC  Curve 
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pass.  If  this  were  true,  than  the  acceptability  of  the  entire 
lot  might  often  be  based  on  whether  or  not  this  scenario  could 
be  passed  by  the  required  number  of  missiles.  To  better 
understand  this  issue  MICOM  conducted  a  simulation  study  to 
determine  to  what  extent  each  scenario  was  more  difficult  than 
another.  The  results  of  this  simulation  are  depicted  below  in 
Table  5-1,   Closed  Loop  Electronic  Tests. 

TABLE  5-1,   CLOSED  LOOP  ELECTRONIC  SIMULATION  TESTS 


Scenario 

Results* 

#1 

+  1 

#2 

(-19) 

#3 

0 

#4 

0 

#5 

(-21) 

#6 

+  9 

*  Results  are  given  in  millimeters 
Each  scenario  was  conducted  100  times  and  the  Results  column 
in  the  table  displays  the  difference  in  millimeters  between 
each  scenario  and  scenario  #3.  The  results  are  given  in 
Circular  Error  Probability  (CEP) .  Since  the  intended  target 
of  the  Longbow  HELLFIRE  missile  is  a  tank  roughly  2  meters 
wide  by  7  meters  long  by  2  meters  high,  the  maximum  deviations 
indicated  in  this  table  between  each  scenario  is  not  very 
significant . 
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3.   Comparison. 

When  comparing  the  two  plans  it  appears  that  the  HWIL 
QALVT  Plan  places  the  contractor  at  greater  risk  then  the 
original  FTB.   This  comparison  is  given  in  Table  5-2. 

TABLE  5-2,   COMPARISON  OF  PROBABILITY  OF  ACCEPTANCE. 


4-4-2  FTB  Plan 

6-6-3  HWIL  Plan 

STAGE  #1 

0.78 

0.70 

STAGE  #2 

0.92 

0.84 

STAGE  #3 

0.98 

0.94 

OVERALL  TOTAL: 

0.97 

0.92 

The  information  depicted  in  Table  5-2  was  based  on  the 
assumption  that  the  missiles  being  tested  in  either  of  the 
plans,  do  in  fact,  meet  the  contract  system  reliability 
requirement  of  0.949. 

In  fact,  the  contractor  has  noted  this  increased  risk 
and  has  contacted  the  Longbow  HELLFIRE  Program  Office  to 
register  his  concern.  He  has  stated  that  he  is  being  placed 
at  greater  risk  than  his  previous  exposure  under  former 
HELLFIRE  contracts  utilizing  FTB.  The  PMO's  response  has  been 
to  take  the  issue  under  consideration  until  such  time  as  the 
entire  HWIL  simulation  plan  solidifies  to  a  point  where  a 
better  determination  on  this  issue  can  be  made.   Both  plans 


9  Dick,  R.,  "Binomial  Sampling  Plans  Compaison",  Brief, 
January  1990. 
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satisfy  a  requirement  to  accept  the  lot  with  probability 
greater  then  0.92  when  actual  missile  reliability  is  0.94. 
Except  for  the  testing  cost  factor  ratio  of  approximately  7:1 
in  favor  of  HWIL,  both  plans  would  be  acceptable  for 
Government  use. 

The  only  other  major  area  of  comparison  is  the 
quantity  of  the  sample  size.  In  both  cases  the  quantity  used, 
was  driven  by  the  desire  to  make  a  determination  of  lot 
acceptability  after  the  least  amount  testing  and  with  the 
fewest  number  of  missile  test  articles  involved.  The 
structure  of  each  plan  is  basically  the  same  except  for  the 
additional  requirement,  under  the  HWIL  QALVT,  that  each 
missile  must  complete  two  out  of  three  runs  successfully  for 
all  six  scenarios.  This  reason  might  give  the  impression  of 
a  significant  difference.  However,  the  likelihood  of  a 
missile  passing  one  of  the  six  scenarios,  given  it  has  passed 
one  or  more  of  the  other  scenarios  is  larger  than  it  would  be 
if  this  information  were  not  available.  That  is,  the  outcomes 
of  the  scenario's  tests  are  not  statistically  independent.  In 
addition,  the  repeated  three  trials  for  each  component  are  not 
statistically  independent  because  it  is  the  same  missile  under 
test  in  all  three  trials.  This  is  based  on  the  premise  that 
HWIL  testing  involves  testing  only  the  electronic  hardware  of 
the  guidance  and  control  systems  and  the  system  software  that 
run  these  components.  It  stands  to  reason  that  if  these  two 
components  perform  reliably  the  first  time,  they  are  more  than 
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likely  to  continue  to  perform  appropriately  for  all  the 
remaining  trials  within  that  scenario. 

Thus,  the  trials  and  scenarios  are  not  truly 
independent  of  one  another  as  is  assumed  in  the  plan.  This 
independence  feature  was  used  to  compute  the  acceptance 
probability  of  92%.  If  this  factor  of  dependence  is  accounted 
for  in  a  fairly  conservative  manner,  the  true  acceptance 
probability  is  closer  to  94%.  While  this  number  is  closer  to 
the  FTB  number,  it  may  still  represent  additional  risk  to  the 
contractor.  The  extent  of  this  additional  risk  cannot  be 
assessed  without  actual  test  data. 

The  real  issue  here  is  not  the  number  of  scenarios, 
number  of  trials,  or  pass/fail  criteria,  but  the  structure  of 
the  test  in  general.  Many  different  test  structures  can  be 
designed  and  their  probability  of  lot  acceptance  computed  as 
can  be  seen  in  Table  5-3  Sampling  Plan  Comparison. 

However,  what  comes  to  light  in  the  analysis  of  these 
test  sampling  plan  schemes,  is  that  they  all  are  based  on  the 
assumption  of  multi-stage  lot  acceptance  criteria,  which 
assumes  that  there  is  a  underlying  desire  to  make  a 
determination  after  the  least  amount  of  testing,  because  of 
the  high  cost  that  would  normally  be  involved  if  this  were 
true.  This  is  not  the  case  for  HWIL  simulation  tests.  As 
described  earlier  in  Chapter  II,  the  STAF  facility  can  test 
about  20  +  missiles  a  month  at  the  same  cost  to  do  the  15 
missiles  in  the  plan. 
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TABLE  5-3,   SAMPLING  PLAN  COMPARISON, 


Sampling  Plan 
(N  -  Ace  -  Re j ) 

Overall  Probability  of  Acceptance 
given  a  Reliability  of  0.94 

20-2-3 

0.885 

10-1-2 

0.882 

9-0-2 
9-1-2 

0.762 

8-0-2 
8-1-2 

0.799 

7-0-2 
7-1-2 

0.836 

6-0-2 
6-1-2 

0.872 

**  6  -  0  -  2  ** 
6-1-3 
6-2-3 

0.92  (0.937) 

7-0-2 
7-1-3 
7-2-3 

0.891 

8-0-2 
8-1-3 
8-2-3 

0.858 

Fly  To  Buy 

4-0-2 
4-1-3 
2-2-3 

0.971 

Note:  The  **  6  -  0  -  2  **  indicating  the  current  test  scheme. 
The  6-6-3  multi-stage  HWIL  plan  still  requires  all  15  missiles 
to  be  delivered  to  the  Government  since  there  is  no  way  of 
determining  before  hand,  at  what  stage  the  decision  will  be 
made. 


63 


C.   PLIGHT  TESTING. 
1.   Introduction. 

Flight  testing  beyond  EMD  and  LRIP  for  the  Longbow 
HELLFIRE  missile  will  consist  of  firing  4  production  missiles 
from  static  launchers,  at  Eglin  AFB,  Fl .  One  missile  will  be 
chosen  from  each  batch  of  lot  samples  for  each  quarter  year. 
These  missiles  will  have  already  passed  their  particular  lot 
acceptance  test  in  the  STAF  facility  prior  to  there  selection 
for  firing.  Once  selected  they  will  be  transported  to  EAFB 
and  prepared  for  firing  against  one  of  the  six  scenarios  used 
in  the  lot  acceptance  plan.  This  flight  testing  is  expected 
to  provide  a  quality  check  on  areas  of  the  missile  system  not 
tested  under  either  of  the  earlier  open  and  closed  looped 
testing  performed  at  RSA.   These  areas  will  include: 

1)  Rocket  Motor  and  ignition  device. 

2)  Electronic  Safe  and  Arm  device  (ESAF) . 

3)  Warheads  and  detonation  train. 

4)  The  Missile  on  board  batteries. 

5)  Other  Flight  related  Components  not  tested  via  HWIL 
testing. 

Flight  testing  will  also  serve  as  a  quality  check  and 
source  of  additional  information  on  the  HWIL  Simulation 
process  since  all  flight  test  missiles  will  be  drawn  from  HWIL 
specimens.  However,  other  than  verifying  that  a  test  missile 
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hits  or  misses  the  intended  target,  there  is  little  means  of 
effectively  gathering  useable  data  from  these  flight  test 
missiles.  None  of  these  missiles  will  be  electronically 
instrumented  or  outfitted  with  telemetry  devices  that  could 
transmit  guidance,  trajectory,  and  flight  data  back  to  a 
ground  station  for  post  flight  analysis.  The  only  data 
gathering  mechanism  is  the  high  speed  filming  from  range 
tracking  devices  and  the  point  of  impact. 

While  the  need  for  this  testing  is  not  being 
questioned,  some  very  real  and  important  questions  do  exist 
that  could  potentially  adversely  effect  the  HWIL  lot 
acceptance  process. 

2.   Question  Areas. 

a.  Flight  Test  Significant. 

As  mentioned  above,  one  missile  a  quarter  will  be 
test  flown  at  EAFB.  At  this  time  there  is  no  particular 
plan/guideline  to  cover  or  evaluate  these  flight  test 
missiles.  The  sole  evaluation  criterion  will  be  whether  the 
missile  hits  or  misses  the  intended  target.  Because  these 
missiles  are  drawn  from  samples  that  have  already  passed  the 
HWIL  simulation,  any  failures  during  these  flight  tests  will 
generate  a  series  of  unanswerable  questions  that  will  be 
difficult  or  impossible  to  resolve.   Such  questions  as: 

1)  What  specifically  went  wrong  with  the  missile? 

2)  Should  it  have  passed  the  HWIL  Simulation  and  then 
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failed  to  hit  the  actual  target;  i.e.,  is  there 
a  potential  defect  in  the  simulation  testing? 

3)  What  do  these  flight  test  results  indicate  about 

the  rest  of  the  missiles  in  this  particular  production  lot? 

4)  What  needs  to  be  fixed  and  how? 

The  first  of  these  questions  "What  specifically 
went  wrong?",  may  never  be  answered.  Since  there  is  no 
instrumentation,  only  obvious  failures  such  as  the  rocket 
motor  not  igniting  or  the  warhead  failing  to  detonate  could  be 
investigated  with  any  certainty.  Less  obvious  problems 
involved  in  target  misses  may  never  be  resolved. 

"Should  it  have  passed  the  HWIL  simulation  and 
then  missed  the  target?".  Because  post  flight  data  are 
limited,  this  question  may  never  be  answered  satisfactorily. 
In  fact,  this  particular  question  will  likely  serve  to 
generate  additional  speculation  in  other  areas  such  as 
transportation  and  mishandling. 

The  third  question  presents  quite  a  major  dilemma, 
because  the  minimum  number  of  missiles  necessary  to  answer 
this  question  is  the  same  quantity  that  is  indicated  in  the 
former  FTB  plan;  namely  four  missiles  per  lot.  Since  there 
will  be  only  one  missile  fired  from  one  of  the  three  lots 
accepted  during  that  period,  no  meaningful  inference  can  be 
made  about  other  missiles  in  the  lot  using  the  one  flight 
test . 
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Finally,  the  last  question  on  the  list  represents 
the  fall  out  from  the  other  three.  This  question  can  only  be 
answered  after  decisions  are  made  relating  to  the  other  three 
questions.  However,  if  some  prior  thought  is  not  given  to 
these  questions  before  hand  and  a  course  of  action  taken,  then 
this  area  will  have  to  be  dealt  with  after  the  fact  when  it 
will  be  potentially  more  difficult  and  costly  to  resolve. 
Jb.  HWIL  Feedback. 

It  is  understood  that  prior  to  any  actual  lot 
acceptance  testing  being  conducted  in  the  STAF  facility,  the 
process  used  by  this  facility  will  have  to  be  validated  to  a 
certain  level.  Once  that  level  is  achieved,  it  will  become 
difficult  to  improve  on  the  process,  because  the  source  of 
data  necessary  to  accomplish  this  activity;  namely,  telemetry 
instrumented  test  missiles  will  no  longer  be  available.  It 
stands  to  reason  that  the  more  information  one  has  concerning 
a  population  of  missiles  (data  base)  the  greater  fidelity  the 
modeling  and  simulation  of  that  missile  will  be.  The  same 
holds  true  here.  But  as  of  now,  there  are  no  plans  to  build 
additional  telemetry  rounds,  since  these  missiles  are 
configured  with  a  telemetry  package  in  place  of  a  warhead. 

Any  retro- fitting  of  production  missiles  is  not 
desirable  from  two  perspectives,  1)  it  will  not  be  truly 
representative  of  the  rest  of  the  lot  it  was  produced  from, 
once  retro- fitted,  and    2)   the  cost  involved  makes  it 
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impractical . 

Another  alternative  available  to  the  Government  to 
improve  the  HWIL  simulation  process,  would  be  to  allow  the 
firing  of  several  of  the  missiles  that  have  been  determined  to 
be  not  acceptable  by  the  closed  loop  test  only.  They  could  be 
taken  to  the  flight  test  range  and  fired  for  the  purpose  of 
gathering  additional  information  and  insight  into  the  process 
itself.  Such  a  program  would  help  to  verify  the  accuracy  of 
the  simulation.  This  testing  program  may  not  be  advisable  for 
safety  reasons,  but  the  ability  to  confirm  the  good  as  well  as 
the  bad  missiles  would  be  a  valuable  feedback  tool  to  improve 
the  process  as  a  whole. 

D.   VALIDATION  AND  CERTIFICATION 

1.   Background  and  Requirements. 

HWIL  simulations  have  been  used  on  several  occasions 
to  help  make  lot  acceptance  determinations.  Such  systems  as 
the  Army's  TACMS  and  Pershing  missiles,  and  the  Navy's 
Tomahawk  Land  Attack  Missile  (TLAM)  have  all  used  HWIL 
simulations  in  some  form,  but  not  as  the  basis  for  a  lot 
acceptance  process.  In  each  case  the  validation  of  these 
simulations  poses  an  all  together  different  problem  for  each 
system  for  different  reasons.  Validation  is  concerned  with 
the  fidelity  of  the  model  and  its  accurate  portrayal  of  the 
real  life  event.  The  procedures  required  to  accomplish  this 
validation   are   not   always   the   same   because   system 
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characteristics  and  requirements  differ  so  radically  from  one 
to  another.  The  validation  process  must  be  attacked  on  a  case 
by  case  basis  tailored  for  the  unique  characteristics  of  the 
system  and  the  end  product. 

As  of  now,  there  is  no  set  plan  to  accomplish  this 
task,  but  several  factors  will  be  key  to  its  successful 
completion.  They  are  1)  the  determination  of  the  accuracy  of 
the  simulation  to  its  real  life  counterpart,  2)  the  inherent 
reliability  of  the  simulation  to  function  properly  and 
accurately  portray  the  proper  environment  without  inducing 
additional  errors  due  to  defects,  anomalies  or  modeling  errors 
into  the  process.  In  both  of  these  cases,  any  factor  that 
would  decrement  the  outcome  of  this  process  from  the  real  life 
environment  that  the  missile  would  be  subjected  too,  could  be 
perceived  by  the  contractor  as  additional  risk  to  him  and  be 
a  basis  for  an  increase  in  costs  and  missile  price.  For 
example,  suppose  the  simulation's  accuracy  was  determined  to 
be  95%  faithful  to  the  actual  system's  characteristics.  What 
does  this  5%  difference  represent  to  the  two  parties  involved? 
Will  it  impact  on  the  determination  of  the  acceptability  of 
the  lots  under  inspection?  2)  If  the  simulation  software  were 
deemed  to  work  properly  and  produce  an  accurate  result  99%  of 
the  time,  the  1%  of  the  time  that  it  did  not  function  properly 
could  be  perceived  by  the  contractor  as  an  additional  1%  of 
risk  he  was  assuming  to  perform  lot  acceptance.  This  would 
represent  1%  more  lots  that  might  not  be  accepted  by  the 
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Government  costing  him  additional  rework  and  handling  that  may 
not  be  justified.  Of  course,  the  opposite  case  is  also 
possible.  The  HWIL  simulation  process  might  be  passing 
defective  missiles. 

No  verification  plan  has  been  formulated  or  drafted  to 
date,  but  several  of  these  problems  have  been  recognized  by 
the  facility  developers.  The  mechanism  requiring  each  sample 
missile  to  pass  only  two  of  three  runs  of  each  scenario  is  an 
example  of  an  attempt  to  reduce  the  prospect  of  a  simulation 
induced  error  effecting  the  overall  acceptance  of  any 
particular  lot. 

These  examples  indicate  an  area  of  the  Longbow 
HELLFIRE  QALVT  plan  that  has  not  been  fully  covered.  One  area 
that  deserves  additional  attention  pertains  to  what  will 
happen  when  a  HWIL  simulation  problem  occurs  during  the 
pursuit  of  lot  acceptance  testing.  Potential  concerns  in  this 
area  would  center  around  the  necessity  of  restarting  that 
lot's  testing,  because  it  may  not  be  determined  if  the  problem 
detected  in  the  missile  under  test  went  undetected  in  all 
prior  missiles.  If  so,  this  would  tend  to  invalidate  prior 
testing  on  missiles  for  that  lot.  This  would  likely  require 
retesting. 

2.   A  Possible  Validation  Model. 

As  mentioned  above,  no  specific  lot  acceptance 
simulation  has  been  through  the  validation  process  to  date. 
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However,  there  have  been  simulations  that  have  been  validated 
and  certified  for  purposes  very  close  to  that  which  would 
serve  the  Longbow  HELLFIRE  system  very  well  as  a  possible 
model  for  their  certification  program. 

This  system  is  the  Navy's  Tomahawk  Land  Attack  Missile 
(TLAM) ,  which  used  a  computer  simulation  called  the 
Interpretive  Simulation  Program  No.  13  (ISP  13.0). 

The  ISP  13.0  simulates  the  TLAM  and  the  environment  in 
which  it  operates.  It  focuses  on  Block  III  and  Pre-Block  III 
Operational  Flight  Software  (OFS)  execution  from  power-up  to 
missile  impact. 

The  plan  that  the  Naval  Air  Warfare  Center,  Weapons 
Division  (NAWCWPNS)  used  to  certify  their  simulation  involved 
a  concept  by  which  they  performed  a  comparison  of  ISP  against 
accepted  standards  outlined  in  the  Simulation  Management  Plan, 
Cruise  Missile  Project  Office,  July  1988.  The  certification 
includes  validation  data  and  verification  data.  The 
comparisons  of  simulation  data  with  flight  data  comprised 
validation.  The  comparison  of  simulation  data  with  previously 
certified  simulation  data  (from  earlier  ISP  versions  10.0, 
11.0,  &  12.03)  comprised  verification.  In  the  case  of  ISP 
13.0  the  validation  is  performed  by  comparing  its  data  with 
selected  test  flight  telemetry  data,  and  verification  is 
performed   by   comparing   against   MVS-RLS10   data.     The 


10  ISP- 11.0  was  updated/certified  under  a  new  name,  MVS 
(RLS-1.1),  in  December  1991. 
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Certification  Test  Matrix  described  in  TABLE  5-4  was  used  to 
perform  this  function  and  included  all  the  runs  for  which  ISP 
13.0  has  been  subjected  to  during  certification  testing. 
TABLE  5-4,   ISP  CERTIFACATION  TEST  MATRIX. 


MISSILE 

PLIGHT 

OPS 

PLATFORM 

REMARKS 

109C 

OTL-39 

BDV014 

SUB-TTL 

• 

109C 

OTL-78 

LDV2  07 

SUB-CLS 

* 

109A 

OTL-45 

BNV153 

SHIP-VLS 

• 

109D 

DT-3 

BDV012 

SHIP-ABL 

* 

109C 

OTL-38 

BDV014 

SHIP-ABL 

• 

109A 

OTL-32 

BNV152 

SUB-TTL 

• 

109C 

OTL-40 

BDV014 

SUB-TTL 

* 

109C 

TBAR-1 

LDV4  00 

SUB-CLS 

•  * 

109C 

TBAR-2 

LDV400 

SHIP-ABL 

** 

109C 

TBAR-3 

LDV400 

SHIP-ABL 

** 

109D 

TBAR-4 

LDV4  00 

SHIP-ABL 

** 

109C 

TBAR-7 

LDV400 

SHIP-VLS 

** 

109D 

TBAR-8 

LDV4  00 

SHIP-VLS 

** 

109C 

DT-1 

LDV400 

SUB-TTL 

** 

109C 

DT-1R 

LDV400 

SHIP-ABL 

*** 

109D 

DT-2 

LDV401.1 

SHIP-ABL 

*** 

109C 

DT/OT-1 

LDV401.1 

SUB-TTL 

*** 

Notes; 


*• 


ISP- 10.0  validation  runs,  resulted  in  provisional 
certification  for  BLK  III  ISP  development. 
ISP- 11.0  verification  test  runs,  resulted  in 
provisional  certification  for  BLK  III  test 
flights. 
***  ISP  13.0  validation  and  verification  runs,  to  be 
completed  for  full  certification. 

The  following  parameters  were  plotted  for  ISP  13.0 

runs  from  the  test  matrix  above  and  compared  with  flight 
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telemetry  data  for  validation,  and  MVS  (RLS-1.1)  simulation 
for  verification.   The  parameters  were: 

•  Latitude  versus  Longitude. 

•  Inertial  Altitude  versus  time. 

•  Roll  versus  time. 

•  Yaw  versus  time. 

•  Roll  rate  versus  time. 

•  Pitch  rate  versus  time. 

•  Yaw  rate  versus  time. 

•  Mach  command  versus  time. 

•  Mach  versus  time. 

•  North  velocity  versus  time. 

•  East  velocity  versus  time. 

•  Vertical  velocity  versus  time. 

•  Normal  acceleration  versus  time. 

•  TOA  error  versus  time. 

•  Air  temperature  versus  time. 

•  Static  pressure  versus  time. 

•  Dynamic  pressure  versus  time. 

A  typical  plot  that  was  developed  to  perform  the 
comparison  function  for  the  DT1R:  ISP  13 . 0 ... Inertial 
Altitude,  Vertical  Velocity,  and  Normal  Acceleration  vs 
Telemetry  can  be  seen  in  the  right  hand  column  at  Figure  11. 
The  solid  line  in  these  graphs  represents  the  simulation 
results.   The  dotted  lines  are  the  actual  missile  telemetry 
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Figure   11     Telemetry  vs  Simulation  Comparison 
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readings  overlaid  on  the  computer  simulation  graph.  Figure  11 
also  depicts  an  example  of  the  type  of  close  match  the 
Simulation  Management  Board  would  be  looking  for  during  the 
validation  and  certification  process.  A  perfect  match  would 
be  both  graphs  laid  one  upon  the  other. 

The  closer  the  simulation  results  (dotted  lines)  match 
the  missile  telemetry  data  (solid  lines)  the  more  accurate  the 
simulation  would  be  to  the  true  life  event  it  was  modeling. 
If  this  were  true  for  the  majority  of  the  parameters  being 
modeled,  and  the  results  were  consistent  over  a  number  of  runs 
(simulation  matched  with  flight  telemetry  data) ,  then  it  could 
be  reasonably  assumed  that  the  model  was  an  accurate  portrayal 
of  these  events.  This  is  the  approach  that  the  TLAM  Program 
Office  is  pursuing  for  their  ISP  model  for  the  TLAM. 

While  the  missile  system  attributes  may  be  different 
for  Longbow  HELLFIRE  system,  the  basic  procedure  to  perform 
the  certification  process  would  be  the  same.  This  process  is 
one  of  matching  a  number  of  simulation  runs  with 
their  corresponding  flight  telemetry  data,  and  then  assessing 
the  results  between  these  two  runs. 

As  with  the  TLAM  Program,  an  independent  Software 
Management  Board  (SMB)  could  be  established  to  review  the  data 
and  determine  whether  a  certification  of  the  simulation  could 
be  rendered. 

To  implement  and  control  this  task,  a  weapon  specific 
simulation  certification  plan  and  process  would  have  to  be 
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developed.  An  example  of  the  process  that  TLAM  used  in  their 
certification  effort  can  be  seen  in  Appendix  C.  This  process 
could  easily  serve  as  a  guideline  for  Longbow  HELLFIRE's 
certification  effort  with  some  modifications. 
3.   Summary. 

Validation  and  Certification  of  the  HWIL  simulation 
represents  a  significant  challenge,  but  one  that  can  be 
solved.  The  TLAM  example  is  just  one  possible  solution  to 
this  challenge.  Further  insight  into  this  area  can  be 
obtained  through  a  new  book  on  the  process  of  validating 
simulations  entitled:  "Simulation  Validation,  A  Confidence 
Assessment  Methodology".  This  book  provides  a  generic 
foundations  for  describing,  assessing,  and  structuring  the 
simulation  validation  process  and  many  of  the  problems  areas 
that  may  be  encountered  along  the  way11. 


11  Knepell,  P.L.,  Simulation  Validation,  IEEE  Computer 
Society  Press,  Los  Alimitos,  California,  1993. 
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VI    CONCLUSION 

A.  INTRODUCTION. 

After  analyzing  all  the  pertinent  information  and  data 
regarding  the  Longbow  HELLFIRE  HWIL  lot  acceptances  plan, 
several  areas  have  been  identified  that  may  require  additional 
attention  prior  to  implementation  of  this  test  plan.  These 
areas  will  form  the  basis  of  this  chapter  and  will  be 
discussed  individually  in  order  of  importance. 

B.  BASIS. 

In  comming  to  a  conclusion  several  areas  stood  out  as 
being  of  particular  importance  to  implementing  the  final  HWIL 
acceptance  plan.   These  areas  are: 

1)  The  role  of  Longbow  HELLFIRE  missile  flight  testing,  as 
it  pertains  to  HWIL  lot  acceptance  testing. 

2)  The  nature  and  extent  of  the  validation  and 
certification  of  the  HWIL  simulation. 

3)  How  improvements  in  the  process  will  be  achieved. 

4)  Implications  and  future  use  by  other  systems. 

5)  Stockpile  Surveillance. 

1.   The  Role  of  Flight  Testing. 

The  role  of  Longbow  HELLFIRE  missile  flight  testing 
presents  one  of  the  biggest  potential  dangers  to  the  program 
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if  not  addressed  satisfactorily.  Here  you  have  a  case  where 
an  insufficient  quantity  of  missiles  that  will  have  all  passed 
the  HWIL  simulation  test  are  taken  to  the  range  and  fired  at 
a  given  scenario  target  with  no  attempt  at  either  separating 
the  results  from  the  simulation  or  qualifying  them  with 
respect  to  the  simulation.  In  both  cases,  given  the  small 
quantity  (one  missile  flight  per  quarter)  it  will  be  extremely 
difficult  to  draw  any  kind  of  meaningful  conclusion  if  a 
missile  should  fail  to  hit  its  target.  Increasing  the  number 
of  flight  tests  per  quarter  for  the  first  fiscal  year  should 
facilitate  the  validation  of  the  HWIL  simulation.  Failure 
analysis  would  be  limited  to  trajectory  and  film  reviews  of 
the  flight,  and  the  specter  of  whether  or  not  the  simulation 
was  passing  on  defective  missiles  could  not  be  easily 
answered.  The  intended  role  of  these  flights  was  to  prove  out 
that  non- tested  areas  of  the  missile  such  as  the  warhead  and 
rocket  motor  will  and  do  in  fact  function  properly.  The 
probability  of  detecting  faults  in  these  non- tested  areas  is 
not  very  large  even  when  they  occur  in  a  significant 
proportion  (say  15%)  of  the  population  of  missiles  if  only 
four  missiles  are  sampled  per  year.  The  probability,  Pd,  of 
detecting  such  a  fault  (if  at  least  one  of  4  missile  fail) 
when  15%  of  the  missiles  are  defective  is: 

Pd  =  1  -  (0.85)4  =  0.478 
The  number  of  tests,  n,  needed  per  year  to  assure  that  the 
estimate,  R,  of  the  true  successful  flight  reliability  R  is 
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within   one- fourth   of   a   standard   deviation   of   R   with 
probability  0.9  0  is  approximated  by: 

n  =  (1.645)2  *  42  =  43 
To  be  within  one-half  standard  deviation  of  R  with  probability 
0.90  requires  about  10  missiles  per  year.  If  R  =  0.85,  the 
standard  deviation  is  (0 . 85) *  (0 . 15)  =  0.1275.  To  be  within 
one- fourth  a  standard  deviation  equates  to  R  differing  from 
0.85  by  +  or  -  0.03,  with  probability  0.90.  To  be  within  one- 
half  a  standard  deviation  equates  to  R  differing  from  0.85  by 
+  or  -  0.06,  with  probability  0.90. 

2.  Validation  and  Certification. 

Validation  and  certification  should  be  concerned  with 
what  can  be  reasonably  tested  and  substantiated  within  the 
HWIL  simulation  test  and  what  cannot.  Knowing  these  limits 
and  limitations  will  help  determine  what  modifications  to  the 
entire  procedure  are  warranted.  As  described  earlier  in 
Chapter  V,  there  are  existing  programs  that  have  successfully 
validated  and  certified  their  simulations.  The  TLAM  example 
is  just  one  method  that  has  been  performed  that  fulfills  this 
requirement,  and  could  be  used  as  a  model  for  the  Longbow 
HELLFIRE  validation  and  certification  process. 

3.  Improvements. 

It  stands  to  reason  that  this  is  only  the  first  step 
in  what  will  be  an  eventual  dependence  on  HWIL  simulations  for 
lot  acceptance.  Most  improvements  to  the  process  will  only  be 


79 


discovered  after  the  facility  is  operational  and  certain 
amount  of  experience  has  been  gained.  With  this  in  mind,  some 
attention  should  be  given  to  the  area  of  planning  for  process 
improvements  now  to  more  easily  take  advantage  of  better 
methods  as  they  become  available. 
4.   Future  System  Application. 

Other  Army  programs  have  been  approached  about  using 
the  STAF  facility  on  a  time  sharing  basis  and  using  HWIL 
simulation  to  reduce  testing  costs.  This  program  would 
include  Non-Line  of  Sight  (NLOS) ,  the  earlier  HELLFIRE  II, 
JAVELIN  man-portable  anti-tank  missile,  Brilliant  Anti-tank 
(BAT)  ,  and  CORPS  SAM.  The  key  factor  here  is  the  need  to 
highlight  the  necessity  of  early  planning  to  accomplish  this 
task. 

Longbow  HELLFIRE  has  already  demonstrated  that  it  is 
much  easier  to  design  features  into  the  missile  that  lend 
themselves  to  aiding  the  ease  of  HWIL  testing,  than  attempting 
to  correct  these  after  the  missile  design  has  been  locked  in. 
In  the  case  of  Longbow  HELLFIRE,  the  move  to  HWIL  simulation 
meant  determining  an  after  the  fact  means  to  attach  the 
simulation  hook  up  cable  through  a  sealed  access  panel  in  the 
missile  body.  In  addition  numerous  signal  and  interface 
problems  and  modifications  were  needed.  These  requirements 
would  have  been  easier  to  resolve  if  they  had  been  included  in 
the  original  system  design  to  accommodate  an  HWIL  approach. 
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Future  systems  should  investigate  this  area  during  the  design 
phase,  so  that  HWIL  critical  features  can  be  incorporated  into 
the  design. 

5.   Stockpile  Surveillance. 

Although,  not  a  part  of  the  Longbow  HELLFIRE  QALVT 
program,  an  area  that  warrants  attention  here  because  of  its 
obvious  impact,  is  stockpile  surveillance.  Normally  at  a 
predetermined  time  in  the  storage  life  of  missiles,  a  certain 
quantity  of  missiles  are  selected  at  random  from  a  given  lot, 
inspected  and  test  fired.  This  process  helps  assess 
deterioration  characteristics  of  stored  missiles  and 
facilitate  the  identification  of  problems  that  can  be 
corrected  before  they  continue  to  degrade  the  rest  of  the 
stockpile.  HWIL  simulations  may  be  useful  for  this  type  of 
testing  and  may  provide  substantial  savings  for  the 
Government.  In  addition,  some  failures  could  in  all 
likelihood  be  traced  to  a  specific  component  and  corrected; 
whereas  with  flight  testing,  the  cause  of  the  failure  might 
never  be  determined.  With  failure  modes  identified,  it  would 
be  that  much  easier  to  implement  corrective  fixes  or 
incorporate  them  in  future  lots  or  designs. 
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VII         AREAS    OF    CONCERN 

A.  GENERAL. 

The  purpose  of  this  chapter  is  to  identify  potential  problem 
areas  that  have  so  far  come  to  light  in  this  thesis.  It  is 
hoped  that  these  suggestions  are  viewed  as  an  aid  to  solving 
these  problems  and  not  as  unsolicited  criticism  to  a  process 
that  is  already  well  along  its  way  to  providing  substantial 
benefits  for  the  Army  and  DOD. 

B.  TEE  PROBLEMS  AREAS. 
1.   Flight  Testing. 

Flight  testing,  as  it  is  currently  planned  in  the  Longbow 
HELLFIRE  program,  does  not  appear  to  provide  significant  added 
value  to  the  QALVT  program.  The  original  purpose  of  this 
testing  was  to  provide  a  basis  to  checkout  other  component 
areas  of  the  Longbow  HELLFIRE  missile  system  that  were  not 
tested  during  the  HWIL  simulation  process.  However  it  suffers 
from  certain  major  deficiencies  that  will  hamper  obtaining  any 
meaningful  results.  Because  only  one  missile  out  of  a 
possible  250-400  missiles  produced  during  a  quarter  will  be 
fired,  it  will  be  extremely  hard  to  draw  any  solid  conclusions 
from  any  failures  that  may  occur  during  these  test  flights. 
Since  these  test  articles  will  be  missiles  accepted  straight 
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out  of  the  STAF  testing  facility,  failures  after  passing  in 
those  tested  areas  will  be  hard  to  explain  if  not  attributed 
directly  to  some  shortcoming  in  the  HWIL  simulation  test 
itself. 

There  are  no  easy  answers  to  this  problem.  Flight 
testing  is  absolutely  essential  to  the  process  of  determining 
whether  the  warhead,  rocket  motor,  safe  and  arming  functions, 
as  well  as  the  functions  that  were  tested  via  HWIL  are 
functioning  properly  under  actual  operational  conditions.  The 
crux  of  this  issue  is  how  to  relate  these  tests  to  the 
simulation  in  a  meaningful  manner. 
2.   Feedback. 

The  HWIL  simulation  was  developed  and  constructed  with 
the  aide  of  an  extensive  amount  of  actual  fight  test  data  and 
simulation  work  within  the  MICOM  Laboratories.  Because  of 
this  effort  the  people  involved  were  able  to  develop  this 
process  to  the  point  that  it  can  be  used  for  lot  acceptance 
testing.  This  effort  has  been  done  largely  in  parallel  with 
the  current  Longbow  HELLFIRE  EMD  program.  It  has  taken 
advantage  of  the  numerous  instrumented  missile  flight  tests  to 
perfect  the  HWIL  simulation  process.  But  now  as  EMD  ends  and 
production  begins,  access  to  additional  sources  of  future 
flight  data  will  not  be  available  for  process  improvement.  As 
discussed  earlier  only  production  configured  missiles  will  be 
flight  tested  in  the  future.    This  will  not  provide  any 
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substantial  feedback  into  the  HWIL  simulation  process  so  that 
it  can  be  further  refined  and  improved. 
3.   Utilization  By  Future  Systems. 

As  described  earlier,  a  whole  generation  of  future 
tactical  and  operational  level  missile  systems  could  all 
benefit  from  this  type  of  lot  acceptance  testing.  The 
majority  of  these  missiles  already  surpass  the  MICOM  QALVT 
criteria  or  are  very  close  to  it.  The  major  concern  here  is  | 
that  these  systems  need  to  make  decisions  about  simulation 
testing  now  while  still  in  their  developmental  stages.  It  has 
already  been  discovered  that  as  these  systems  increase  in 
complexity  the  need  to  design  in  access  and  compatibility  with 
simulation  testing  up  front  is  crucial.  Delaying  this 
activity  will  only  create  additional  design,  interface,  and 
cost  problems. 

While  this  is  not  a  Longbow  HELLFIRE  specific  problem, 
the  people  matrix  from  the  Product  Assurance  Directorate  to 
the  program  office  should  help  disseminate  this  information  to 
the  rest  of  the  community  at  large  so  that  these  insights  can 
be  incorporated  into  the  next  generation  of  missile  systems. 
Lot  acceptance  planning,  methods,  processes,  and  procedures 
need  to  be  identified  and  planned  for  in  the  early  stages  of 
development. 
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APPENDIX  A   LONGBOW  HELLFIRE  PROGRAM  INFORMATION 


DEVELOPER: 


PHONE : 
CONTRACTOR (S) 

CONCEPT: 


TECHNOLOGY : 

LETHAL 
MECHANISM: 

PRIMARY 
TARGET : 

PERFORMANCE 

STATUS : 

FUNDING: 

TIMETABLE : 

SOURCE 
SUPPORT: 


PEO  Tactical  Missiles;  Project  Manager, 
Air- to-Ground  Missile  Support  Project 
Office,  ATTN:  SFAE-MSL-HD,  Redstone 
Arsenal,  AL.  35898-5610 

(205)  876-8367 

Martin  Marietta,  Rockwell  International, 
and  Westinghouse. 

Longbow  is  a  MMW  fire-and- forget  Hellfire 
missile  system.   It  has  an  improved  range 
over  the  standard  Hellfire,  and  adds  the 
capability  for  all  weather  operations. 

MMW  fire  control  radar  and  seeker. 


Shaped  charge  warhead. 

Armored  vehicles. 

Classified 

Currently  in  the  Engineering  and 
Manufacturing  Development  (EMD)  phase. 

Production  should  begin  in  late  FY94. 
Project  Manager,  Air- to-Ground  Missile 
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APPENDIX  B   STAP  TEST  FACILITY  CAPABILITIES 


OPEN  LOOP  TESTS 

TRANSMIT/RECEIVE  CHARACTERISTICS 

VERIFY  ANTENNA  PATTERNS 

VERIFY  SEEKER  POLARIZATION 

VERIFY  TRANSMITTER  PULSE  SHAPE 

VERIFY  SEEKER  POWER 

VERIFY  TRANSMITTER  CENTER  FREQUENCY 

VERIFY  TRANSMITTER  PRF 

VERIFY  TRANSMITTER  FREQUENCY  STEPS 

VERIFY  TRANSMITTER  BANDWIDTH 

VERIFY  CROSS -POLARIZATION  ISOLATION 

ANTENNA  CHARACTERISTICS 

VERIFY  ANTENNA  PATTERNS 

VERIFY  BEAM  WIDTH 

VERIFY  CROSS -POLARIZATION  ISOLATION 

TRACKING  CHARACTERISTICS 

VERIFY  RANGE  DISCRIMINATES 
VERIFY  ANGLE  DISCRIMINATES 
VERIFY  DOPPLER  RESOLUTION 
VERIFY  RANGE  RESOLUTION 

INERTIAL/ CONTROL  CHARACTERISTICS 

VERIFY  FIN  RESPONSE 
VERIFY  FIN  FEEDBACK 
VERIFY  INERTIAL  MEASUREMENT  SENSOR 

CLOSED  LOOP  TESTS 

COMPLETE  SYSTEM  FUNCTIONALITY  TESTS  IN  NONDESTRUCTIVE  TACTICAL 
FLIGHT  SCENARIO 

INCLUDING: 

ACTUAL  TACTICAL  LINE  OF  SIGHT  RATES, 

RANGE  EXTENDED  TARGETS, 

TARGET  DYNAMICS, 

VARIOUS  LAUNCH  CONDITIONS  AND  GEOMETRIES 
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APPENDIX    C       TLAM    SIMULATION    CERTIFICATION    PROCESS 


BnusM  Onmiiiii  Pr»c«»t 
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Figure  2  Certification  Flow  Chart  for  Effectiveness  and  Survivability  Simulation 


CERTIFICATION  PLAN  APPROVAL 

Once  a  simulation  has  been  authorized  for  Tomahawk  effectiveness  ax 
survivability  analyses  the  SMB  tasks  the  TWS  E&S  SCP  with  scheduling  the  simulation  fi 
certification/accreditation.  The  first  step  in  this  process  is  the  submission  of  a  certification  plan 
the  SCP  for  approval.  The  certification  plan  specifies  what  will  be  tested,  how  it  will  be  teste 
and  what  comparisons  will  be  made  to  reference  check  cases  and  other  data  in  the  process 
certification  of  a  simulation.  A  basic  set  of  reference  check  cases  are  detailed  at  the  end  of  tb 
appendix.  These  reference  check  cases  allow  the  responsible  organization  a  media  for  comparisc 
to  demonstrate  that  a  simulation  is  certifiable.  Comparison  of  simulation  data  to  reference  chec 
case  data  should  be  included  as  the  first  step  in  any  certification  plan.  The  responsible  organizatic 
may  propose  any  cost  effective  combination  of  the  below  methodologies  to  complete  certificatio 
In  order  of  decreasing  priority: 

1)  Comparison  to  flight  test  data; 

2)  Comparison  to  other  simulations  whose  certification  basis  is  flight  te 
comparison; 

3)  Comparison  of  data  with  other  certified  simulations; 

4)  Technical  audit  of  simulation  performance; 

5)  Demonstration  of  simulation  capability. 
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